Auto Landscape Material Development: Working Smart on Large Projects

Introduction

Hi there, my name is Lukas Stratmann. I am an Environment Artist originally from Germany. In the past six years, I have worked on the Red Orchestra 2 and Rising Storm franchise. Last year, I was fortunate enough to be able to join Ubisoft in Annecy where I have been a Level Artist on The Division 2. 

When people ask me how I got into 3D I always answer “accidentally”. I have always loved making video games and I learned 3D from colleagues whilst working as a QA Coordinator. Now that I work in the industry, as strange as it sounds, my own art is less important than making a good game. Game Art doesn’t just have to look good: it has to work and allow players to enjoy both the actual game and its visual aspect.

My definition of what makes - my - art good is a much wider picture than just art fundamentals. I love building things, no matter whether they are virtual or physical and I 3D and Environment Art allow me to create spaces and worlds. That brings me closer to my goals of directing the creative effort of making a game. 

Sails of War

For about five years, I have been working on my own side project Sails of War and it has seen many changes. It’s a full project but also my playground to try new things and to grow my knowledge outside of just art. 

The goal was simple: create a scene for our background art, establish a full pipeline and explore any possible problems that might arise. Sails of War is a PvP multiplayer game where each player gets to captain their own tall ship - think War of Warships but with Pirates! 

Spanish Mission: Objectives

Even though I rarely get to work on Sails of War more than during my weekends, it is a full project and with it comes a Jira board. I studied management at university and understand that staying organized is key for finishing a project (yes, I am that one guy in the office who actually likes Jira).

This scene had 4 objectives:

1. Make Buildings and create a shader for it
2. Make props and adapt the Building shader to become a Building/Prop Master
3. Create a Landscape Auto Material with optional hand-drawn layers
4. Spawn vast jungle foliage across large areas and ensure performance 

If you've seen my compilation of WIP screenshots (see below if you haven't) you can see that I tackled the first two objectives early on - it was, in fact, making a new material from scratch based on thoughts and designs I had done back in 2018. 

The new and exciting thing with buildings was the automation of bakes and masks. I had used a mix of unique AO bakes and dirt masks with tiling textures before but never had been able to automate the process. 

Thanks to Jonas Olesen who helped me code a Blender and Substance Automation Toolkit integration, I was able to entirely automate this process which saved literally weeks of baking and map creation.  

Props utilize the same shader - but without a unique overlay texture. I was able to use Marmoset and bake down Megascans assets alongside my original work into very small but high-quality details and props (there is a breakdown of this later in the article).

Auto Landscape Material

In 2018, I had made a regular SSS-based leaf shader and spawned trees on our test map. It was okay but not great. The biggest problem is light baking and lightmaps:
early on, I decided the game was going to use stationary and baked light. We mostly have open water and oceans with all static objects being small and sometimes kilometers away from the camera and quite frankly we just had better FPS numbers using baked light. This, however, poses a problem. 

Spawning trees using the procedural tree system in UE4 can quickly start a struggle with lightmap sizes. Too many objects in a cluster and even a very small lightmap won’t fit on the larger final texture. 

Unreal Engine will give you a warning after baking that the lightmap size is too large. 

An easy answer to that: do as the Kite demo did and turn lights to movable. Make use of all the far distance real-time shadow tips and tricks. 

Now the story could end here. We turn the lights to movable, lose 30 fps but have nice shadows even in the distance. 

This, however, didn’t seem to be smart. With all our foliage being at least 100-200 meters away from the camera you will most of the time only ever see their lowest LOD anyway. Why give up performance for what basically is background filler?

The next immediate thought goes to the amazing Ryan Brucks and his awesome blogpost on octahedral impostors and using HLOD in Fortnite.

“I am probably going to use Imposters” is how I began my Christmas break.

But I also knew that I wanted to use SpeedTree and make use of both the software’s power and its store. Buying a tree, changing it to what I need and exporting it seems like a great and quick way to get things going - and it opens the option to use the software to make unique trees later on. 

Upon exporting the oak tree from SpeedTree I realized the billboards and billboard material it creates were familiar to me in part from Rising Storm 2: Vietnam. The Material will basically only show the plane that is looking at the camera and set the opacity of the other to 0. This means you pretty much always get a nice fade between boards and avoid seeing splits. 

My goal then was to make a material that does not use lightmaps, is able to be used by SpeedTree Billboard and works in all light situations we might run into.

Put short: I was first figuring out what it was I actually needed and wanted. Foliage in Sails of War has specific requirements. For “just” a portfolio scene I would have likely just taken a free Jungle Tree set form the UE4 Marketplace, spawned it and called it a day. This is why I like working on actual projects. Creativity through limitations.

Really from this point onward, I was just taking it step-by-step - again something you can see in the WIP video. I was building it as I went along, always asking how to make it better, sharing my process in places like the DiNusty Empire Discord and incorporating feedback. I knew simply putting the billboard texture on and giving it a brightness multiplier based on the Sun Intensity wasn’t going to look great.
I wanted to give them a full and round feeling. I wanted the outer part to feel “fresh” and younger. I knew that I still needed some sort of light and shadow side. I also needed to make sure intersecting billboards didn’t have a different strength at their seam - or else the illusion breaks.  

Making this material was mainly focused on using the information you already have. We have a normal-map SpeedTree gives us. So we already have information on how light is reflected - even if we can’t use the map in an unlit material. We also have the directional light actor rotation and forward vector. This is something Epic uses in their standard sky material. 

So once we read the information of our light through the Level Blueprint we can feed it into a material parameter collection and use it on the sky, foliage, and all other materials. 

One extra headache is the Procedural Foliage spawn volume I am using to populate the world. All spawned objects have the same origin. So any local space gradients won’t work out of the box. Manipulating this to get the right result can be expensive.

In the end, I am not entirely sure the shader is much cheaper than a regular SSS one - it does, however, not need lightmaps. The only one downside I wasn’t able to resolve is foliage spawned in the shadow of buildings or say in the shadow of mountains. You are not really meant to read lighting information in materials but the grass system seems to be able to read the shadow information so I am sure it is possible to do that somehow. However, it seemed to be out of scope for this particular project. Big thanks to Andrew Bell who was a huge help with all of my odd tech art questions during the entire project. 

Cut off above are the Albedo and opacity crossfade as well as a camera distance-based desaturation - which mimics the fading of contrast at distance.

The most important part is shown and is broken into two areas. One is based on the provided normal map, the other is strictly dependent on the object.

Starting at the lower half and center we first create a bottom-up gradient. This aids in showing the scale and size of the foliage, with lower branches and trunks being darker. The hard part here - as mentioned - was converting what usually would be a simple local space operation to work with the Instanced Foliage. Once we have figured out our “local space” origin at 0,0,0 we can use it to produce the gradient.
We can also use this to create our sphere mask. The sphere mask is a great way to show depth and roundness. It is seamless and helps to bind the billboards together. 

Both these gradients get multiplied with the color to form the emissive output. 

A very big part of creating a good result are shadows. Since we are using an unlit material we don’t have any shadowing or normal maps. We can, however, use the normal map to calculate shadows spots. Finally, we can also use a similar calculation to figure out which part of the mesh is facing the sun.

These too get multiplied with the color. You can see me using the “MCP_World” parameter collection which holes a bunch of values taken from inputs to the Level Blueprint. This includes a later (not shown) additional multiplier for overall brightness. 

This means the end result is usable in all light settings:

Note here: It is rather obvious that our sky still needs the same implementation of automatic brightness adjustments! But you get the point. 

At the end of the day making this shader wasn’t too complex or difficult. Rather, it was a challenge in answering very specific questions and gathering the right information.  

Making Good Art vs Automating the Process

Having a full game as your personal and long term project means lots of limitations and creative ways of getting around problems. This is why I love to work on larger projects rather than just one scene. Sails of War is at the moment a two-men and a part-time team. From its inception, it was obvious that all systems and all design choices need to reflect this manpower. I am a huge advocate for designing solutions that take all and everything into account. You often hear “just make good art”. And that's a good sentiment and it’s great for making singular scenes but for a longer project “good” needs to be redefined and other things become much more important.

Can we produce materials and workflows which allow us to build and rebuild levels faster? Can we develop a system that needs to be set up once but then means we can create all art in a matter of days?

Sails of War puts players and their ship on the water. This means (up next on my roadmap and sprint) we need a good looking and working water material - but this also means making new levels is a matter of drawing obstacles. This means with software like World Machine or Gaea we will create heightmaps and then with well-designed systems in place all we will need to do is plug materials and foliage spawn volumes into this. 

At the end of the day having an Auto Landscape Material just means less headache. Same for the setup of the billboard and soon the clouds. All Materials and all systems are set up so the artist (me) doesn’t have to do the same work over and over again. And that really is the point that brings us back around to automating AO and dirt mask generation. Having to paint the terrain might be quick if you do it once, but if you want to iterate on heightmaps then being able to press a single button and have it all magically work will save a lot of time for making more actual art. We can talk about art fundamentals and how to make good art, and then we can take those learnings and put them into systems that automate the workflow and make life easier.

Tiling Prop Master

The tilling prop master uses this philosophy of automation and well-designed systems. At the end of the day, it’s just a material that overlays a dirt mask and uses a unique UV2 AO mixed with tilling materials. The advantage is in its fine-tuning. With lots of parameters and some switches the artist is able to have fine control over colors, a possible additional world projected moss, dirt and more. This really came in handy for the screenshots. Some of the props use the moss, some don’t. Some buildings have more strength in their normal map, others are lessened. Being able to quickly adjust Material Instances simply makes it easy and being able to use the same Master Material on buildings and Props makes a complex spiderweb of roads into nice and straight highways. The goal here - as with all other materials - is to hide all the complex bits and pieces and make creating a new asset as easy and simple as possible.

Afterword

Sails of War has very complex wave and ocean calculations going on. We have tons of cannonballs flying through the air and we have to all network it. This then is really an exercise of “know where and when to fight your battles”. Players won’t ever be able to get on shore, they won’t see this art up close. That gives me as an artist a great deal of control. Building this scene was also an exercise in creating a pipeline that will allow us to create a new level quickly and without much hassle. Maybe I'm using too many common proverbs but “work smart not hard”. Good design is not only a good front-facing product, but it’s also the backend and the ease of using it. 

There are a million and more things to consider and think about. A simple decision now can have huge ripple effects. The ease of adding new maps, the ease of adding more ships, adding other content, reusability. How much maintenance will it need? How easy will it be to expand into other climates and places?

My job is to answer those questions and make sure we don’t just make a great game, but a game we can easily extend and build upon. I find the most joy not in making the actual art but in creating the entire monster and making sure it’s a well-oiled machine.

I hope you enjoyed this short tour through my latest work and the reasons and design behind it. Thank you for reading.

Lukas Stratmann, Environment Artist

Interview conducted by Kirill Tokarev