Willi Hammes talked about the workflow that allows him to create photorealistic assets, which could be used to build incredible natural environments.
Willi Hammes talked about the workflow that allows him to create photorealistic assets, which could be used to build incredible natural environments.
Introduction
Willi Hammes, self-taught CG generalist working in the film and game industry for over two decades, also founder and owner of CG & VR Studio MAWI United.
During my career, I have been working for different studios in the USA, UK and Germany, on projects like Beowulf, Monster House, Mortal Kombat, Spiderman 2, Everquest 2, X-Men and many others.
Building a business
For quite some time you’ve worked in huge big companies like Blur, Eurocom, Sony Imageworks. Why did you decide to build your own business? It must be so much harder than just going to work?
It simply was time to take the next step. With the shift of games and real-time technology becoming more important and visually appealing, it was the perfect time to put all my knowledge and skills into my own studio.
I love to develop pipelines and workflows and doing this for your own studio is just that more rewarding than doing it for someone else.
Sure it would have been a lot easier to just go to work somewhere, get a paycheck and not worry about all the business side of things. But also important was the fact that I wanted to be in charge of all aspects of production.
High-quality vegetation assets
With MAWI you’ve literally stormed the world with your high-quality vegetation assets. Can you talk a bit about why you’ve chosen this particular field, what makes it so interesting to you?
Well, to be honest, mainly because most trees in games looked like sticks with textured cards as leaves. I thought there was a desperate need for realistic trees with real trunks and roots. Over time the little side project evolved into an entire forest that included all the right details.
And I’m a huge fan of nature. Especially forests and natural landscapes. I have always been very skilled at recreating naturally looking environments in CG. Spending a lot of time outdoors with our little son and dog, walking through the forest, helps tremendously to recreate them in CG.
Challenges
For modern artists, what are the main problems with vegetation and foliage building? It seems that now with the popularity of the photogrammetry and procedural tools you can actually get this content so much faster, but why does it still present a challenge for the specialist?
I think the biggest challenge is knowing what nature really looks like. Get up from your desk and go outside into the forest. How are plants and trees growing? Why do some grow this way and some the other way? Some grow in the shades and some don’t, why?
It’s important to break down nature into patterns and modular blocks. Understand the rules and recreate those in CG. Of course, there are lots of tools that help you to get a realistic plant faster but it’s up to you to know how to put it all together.
Creating foliage is very technical. It is always heavy in terms of hardware resources and quite difficult at times to make it look real but still has it run at an acceptable framerate.
But again the most important thing is to know how to make it look natural.
Redwood Forest
You’ve got quite a lot of packs available, but most recently you’ve shared the Redwood Forest assets, which look absolutely incredible. Could you walk us a little bit through your production process and tell how you usually approach this task? Obviously, you can’t share all of it, but at least some parts of the production would be much appreciated.
The production starts with research. First I decide what pack to build and where to go for capturing references and raw data of the real assets.
I tend to do scout trips to locations of interest to just capture a few photos of possible assets and maybe some test data. Before the actual capture trip, I prep a map and a list of assets that need to be captured. Often we have to go multiple times to get everything or patch up data holes.
Most important for the capture sessions is the right weather. It can’t be raining, it can’t be sunny and can’t be to dark or windy, so perfect are overcast dry days, with enough light to get nice ambient lit objects.
The next step after capturing all the data is structuring and sorting through the material. Remove unusable images and process all the raw photos. The cleaner the source photos the better.
Next, we push all the captured photos through the photogrammetry software. I have setup a little GPU farm at the office to help with processing the massive amount of data. For example, the redwood forest capture trips ended up been 260GB useable raw photo data.
After the data has been processed and initially cleaned up, the high res mesh and raw textures get exported for further preparation and cleanup. We close data holes, fix bad shapes and scale the assets correctly. Also, a low poly retopo mesh is built and we bake the high-res normals and diffuse texture to it We also generate a height map for displacement.
Using these cooked down versions of the assets we do last tweaks in Photoshop. We remove remaining shadows/highlights and generate the different utility textures: specular occlusion, roughness, ambient occlusion, displacement, detail normals and masks.
The final step is importing the assets into UE4. Assigning instanced materials for each asset and setup all the LODs, generate collisions and set up the draw distances. We have developed a few master materials that contain the major functions to process the different input textures.
All of this is a rather fluid workflow where we go back and forth between different steps to tweak the assets until they reach the necessary quality. We keep constantly improving our pipeline and tools, most of the time doing multiple pack developments at once. If we find something useful we tend to roll it back into the other packs to keep them compatible.
Optimization
How do you optimize your content to make it run in the UE4 at a reasonable speed? What are the most efficient ways to cut down on the polycount, make the textures lighter and less unforgiving.
There are options to optimize the content or tweak the engine. We try to keep the shader complexity as low as possible, considering the high quality output, and make sure we have minimal overdraw for all the foliage.
There are different render settings in UE4 for having masked materials processed before all the heavy image passes. Which speeds up rendering of a complex forest with lots of overlapping foliage quite a bit.
We do not cut down on polycount or texture resolution that much. We try to keep it as high as possible and rather build special far distance LODs and LOD shaders that run at a lower complexity at larger distances.
Materials
How do you work on the materials? What are the best ways to build and especially use them in game scenes? How do you make the environment materials look great?
We developed master materials and special material functions that can be used for the majority of assets. This makes it easy to maintain a large number of assets and tweak them all simultaneously.
The actual structure of the materials is quite simple: BaseColor (sRGB), Specular(lin), Roughness(lin), Normal(lin), Subsurface(sRGB) and Displacement(lin). It is just important that each value used, is close to the real life PBR color value to get a consistent output under different lighting conditions.
We also make heavy use of displacement and detail maps to increase the resolution even more when you’re getting super close to the surface. We also build functions to filter details based on distance, to keep the detail and overall structure visible even when further away.
Everything else comes from carefully lighting the example scene. We use those for testing our assets and provide a free learning resource for our customers. It’s pretty much all about balancing light intensities and camera exposure correctly. Knowing a little about classic photography goes a long way too.
Procedural technologies
Could you discuss the way you’ve applied the procedural technologies? The way you’ve painted the forest in UE4 in 3 seconds really made a stir. What’s the algorithm that powers this incredible feature?
This is all based on UE4 internal foliage masking tools. You can have foliage spawn based on landscape layer information and different masks and behaviors. We have set up some rules where the different assets will be spawned, like the surface layer type, density, occlusion and if they can grow on slopes.
Some procedural functions inside the landscape materials are set up to generate the different masks, like the slopes for example. The landscape material generates the majority of the small foliage and detailed ground cover entirely automatic. The large trees and rocks use the landscapes layer information for placement when they get painted.
The cool thing is, when painting the large assets via the foliage painter, the placement is random. So each time you paint the forest it will look different, even when using the same landscape layers.
Future
How do you think the game production technology and environment design, in particular, will evolve in the future. Even today you can create some astonishing environments with Redwood Forest assets in a matter of seconds. And they look very cool. How do you think this will change in the future? Will these tools become cheaper and more accessible? Or maybe will they fade away?
Environment production could rely even more on procedural generation, where the artist only setups some basic rules and the engine automatically picks assets from a library to generate the majority of the environment. So the artist can concentrate on building key gameplay areas and doesn’t need to worry about the surrounding landscape.
Therefore, I think photogrammetry and procedural generation is here to stay for a while. It’s still very hardware intensive and it requires quite a bit of technical knowledge and skill to produce high-quality assets.
But with everything technological, as hardware and software evolve, it will only get faster and easier to do. I think all this will get even more powerful if GPUs and CGI in general move away from a poly based system to something entirely point or volume based.
It will be exciting to see how production will develop in the future.