INFramez allowed us to share some of its R&D results. Homam Bahnassi wrote an article on using machine-learning approaches for baking textures. Smart solution for a challenging task. 

Baked Textures for Games

Baking of textures is a very common technique to pre-calculate rendering attributes to enhance quality and performance in real-time rendering applications.

In games, it is used to bake:

• Lightmaps for enhancing shading by capturing complex lighting (e.g. GI, AO, …etc).
• Texture properties (e.g. Normal maps from high-res models, …etc).

Good quality texture baking depends on many factors. The most important are surface area of the mesh and the output texture’s size. These two factors are measured by what is called “texel density“. For decades, the industry relied on this unit of measurement to determine the quality of baked textures. Although this is technically correct, practically it still misses several other important contributing aspects.

I would claim that texture quality evaluation is based on human perception on the final composition, not just on one abstract number. The challenge with human perception is that it is very hard to quantify, as there are many factors involved (lighting, shadows, color contrast …etc) thus it is very hard to implement a tool that can take into cosideration all the contributing factors to generate the best results that would please human eyes.

Smarter Baking

At INFramez, our pipeline depends on baked textures. So doing R&D on new approaches for this tech is major for our production. Before doing Hyper Void, we were planning for a first-person emotional puzzle/adventure game inspired by Silent Hill. For that project, lightmap baking was a crucial technique for achieving the targeted quality. This made for a good opportunity to invest in researching the previous idea and hopefully materializing a solution for it into a production tool. Due to the nature of the problem, we decided to investigate machine-learning approaches to achieve a computer implementation of this concept. The results from our early prototypes were promising, and with the final prototype the tool was ready for production. Unfortunately, the game project was archived due it is high production costs for our small team. However, the tool was still used in baking lightmaps in Hyper Void.

The following screenshots from Hyper Void shows the panels texture of the spaceship baked with regaluar tool versus the baking with the Machine Learning Enhancment tool.

First image captured from Softimage viewport showing the baked results using the out-of-the-box baking tool:

Second image from in-game showing the same texture baked with this tool:

The main draw-back of the implementation was the processing time. As with most machine learning algorithms, it requires multiple iterations for leaning and evolving. The result from each iteration is used to learn and evolve towards the final bake. Still this drawback was a very acceptable trade for the significant increase in quality and big memory savings on the baked texture sizes. To demonstrate the results of the new technique, I used some of the common test scenes to compare between standard baked textures and textures processed with this new tech we call “machine-learning feature enhancement”.

In addition to the significt quality enhancment, this tool is beneficial for reducing production times. The tool takes care of all art-work assoiciated with the texture-baking process (e.g. uv layouting). Just provide the hi-res mesh and get yourslelf a cup of coffee (maybe two since the processing time is longer).

Please note that the exact same render settings were used when processing the textures for the standard case and the enhanced case.

Future Work

As shown in the videos the algorithm was implemented in Softimage few years ago. In that implementation, the processing time for small scenes is relativaly good (usually few minutes). However, for large complex scenes the processing time can reach several hours as with Sponza. Solving this issue can be done by moving the processing to the GPU to cut the processing time drastically. Luckly, this algorithm is very GPU-friendly and should be a straight-forward job to do.

Since our engine has native support for Softimage, the workflow for using this tool through the editor was straightforward too. But I’d love to get a working version of it to the major commercial game engines too (e.g. Unreal and Unity). If this interests you let us know.

The Algorithm in Action

Baking Lightmap with Global Illumination for Sponza

• Baked Texture Size: 1024×1024
• Number of Triangles: 110,324 triangles
• Processing Time: ~4 hrs

Baking Lightmap with Final Gathering for a City Proxy

• Baked Texture Size: 512×512
• Number of Triangles: 11,406 triangles
• Processing Time: ~10 mins

Baking Ambient Occlusion from Hi-Res to Low-Res Model

• Baked Texture Size: 512×512
• Hi-Res Number of Triangles: 77,280 triangles
• Low-Res Number of Triangles: 10
• Processing Time: ~20 secs

Texture Baking Workflow in Softimage

Preview image by Jimmy Kim