Unreal Engine 4 Stylized Rendering Workflow

Powerful breakdown from Berker Siino, showing some of the intricate details, which will make your stylized environments shine in UE4. Berker has been showing some amazing 3d art over the last couple of years and we’re happy to have a look behind the scenes and see just how much work he puts into his designs.

Please share the article if you find it useful. This helps us a lot.

Introduction

Hi, this is my second interview on 80 Level. My first one can be found here. Big shout out to the guys over at 80 level, you guys are doing a great job for the community!

I am pretty humbled and honored for getting a staff pick on Artstation for my latest piece on a stylized rendering scene in Unreal Engine 4. Thanks a lot to all of you! As a thank you I wanted to share this breakdown with you guys.

I am Berker Siino a freelancer working for clients such as Liquid Development, Airborn Studios, Metalpop Games and some companies in the serious games and simulation space. At the same time, I lately find myself more often in start ups to help getting the art teams up and running with workflows / pipelines and knowledge about 3D art and engines.

My current clients are switching more and more over to work with Unreal Engine 4 and I decided to deepen my knowledge in UE4 with a new personal project. Therefore I did some studies with light, shader networks and rendering in UE4 to be up to date with the latest implementations of the engine. My previous work at Crytek at the beginning of my career enabled me to gain deep technical knowledge about engines and art creation. I learned to become an efficient game developer and problem solver next to being artist. I am a critical thinker and always ask “why”? Understanding the underlying technical basics of the work is one of the most important skills you can have as an artist.

We are in some sort of transition in the games industry these days. Getting assets into real-time applications has become way easier these days and renderings are getting closer to offline rendered quality. A lot of tasks are automated, scanned or recycled from libraries. Tools like

Quixel Suite or Substance Designer/Painter give us the flexibility to be fast and work with a high-quality standard. This trend will keep developing more and more and we will have to make sure to go back to the roots and understand the basics of fine art itself. The tools will soon be as easy to use as holding a pen.

The Reason For The Creation Of The Stylized Unreal Engine 4 Scene

The reason for this scene was to understand the asset creation workflow and rendering pipeline of UE4 and the question of what can I achieve visually in a short time. The approach was to create a lot of assets in a modular way for a bigger bundle of content. I will demonstrate the workflow with Zbrush and Quixel for texturing of three wooden planks which I used as modular pieces to build up certain wood elements or furniture like stairs, terrace, barrels, chairs, etc with one texture set as the base.

Furthermore, I wanted to learn more about dynamic rendering and the use of global illumination in combination with “Light Propagation Volumes” within Unreal Engine 4. My artistic choice for a stylized scene with strong colors and a lot of gradients from GI bounce light was a great way for me to test and experiment with Unreal Engine 4 rendering. It was a welcome change for me to work out a great look without scanned data or a photoreal look. Just the basic workflows for sculpting, modeling and texturing plus the benefit of PBR.

Wooden Assets

This is the highpoly sculpted version of my wooden base kit. No polypaint or texture information, just the highpoly model was created in Zbrush.

I created a lower poly version mesh and wrapped it around the high poly to get surface detail and information.

It would have been even more ideal to fill the remaining UV space with another plank variation or different parts of other assets later on. Keeping things optimized is always a good idea.

For the topology I made sure to have some intersections between the mesh of the plank, so I can have enough topology for bending and deforming the mesh. I kept it in the lower poly range to make sure I have full control and the ability to get rounder or softer shapes with ”flow connect” later on.

A variety of masks got created in Quixel Suite. We will look at that in a moment. Therefore I needed some sort of mesh surface information. For that you can bake out a tangent normal map which is the bare minimum you need, at least in theory. Everything else can be created out of the tangent normal map, of course with less precision and quality than a bake, but to be fair though Ndo2 from Quixel Suite is doing a great job for map generation.

However, in my case I baked tangent normal, ambient occlusion, cavity and created out of the cavity a sharp mask to mask out all indents, wood marks and imperfections.

Since I am working in 3Ds max and also want to preview my normal map results. I make sure to bake out the normal maps to be in sync with 3ds Max’s viewport renderer. The tangent normal map needs following orientation “+X-Y+X”. Remember “Y” is the green channel and can be also flipped later on in the engine or photoshop if something went wrong during the baking process. To preview your bake results in 3D’s Max you can drop a normal map channel into the bump channel of the standard shader in compact mode. Inside the node there will be the option to link the bitmap for the texture to the shader. Make sure to enable the realistic rendering mode in step 3.

Modular Workflow

The great thing about a modular workflow was for me to be able to make first versions of a variety of wooden assets like building parts, terraces or even props really quick. My 3D’s Max scene started to look more and more interesting and enabled me to develop fast on new ideas.

For the barrel (not included in the final renders) I wanted to do an optimized prop asset and simply created a lowpoly mesh on top of the highpoly planks arrangement and rebaked the textures on a better uv layout.That way you can get more variation in the texture or add the metal parts to your uv sheet later on.

The texture to render a projection modifier in 3Ds Max uses the assigned normal map in a material for an asset and mixes it into the bake. This way I got all information plus detail information transferred.

As you can see the meshes are actually really low on poly / triangle count. All the assets you see selected in the scene only have around 47k tris.

Texturing Workflow

The basic workflow here is to use solid colors and masks. You get a lot of great presets with Quixel’s mask generator. This allows you to separate the texturing into sort of tasks or layers for the big picture.

Maybe you want to change the color of the cavity and give it a bit more brownish tint instead of boring black? Furthermore depending on your style you can add more sharpness or blurriness to the output of the mask and control the feel of the color layer. I wanted to create some sort of plasticy but not real looking image. Therefore I did not use a lot of gradient in the albedo map, I used only color and let the PBR values do the rest later on. My gradients got achieved by GI bounce, which was the way I wanted to treat the mesh and ground it with the environment assets that I will build.

I added the first solid color as a base and then build up structure and interesting color variations on top of each other by masking out colors. Here are two .gif files showcasing the texturing progress.

I am adding some brighter color to indicate the chopped off inside of the wood.

Here I am using a cavity map preset to enhance the dark brown color of the indents, scratches and marks of the wood.

House Asset

The scene started to show me how efficient I was able to build these assets in a fast way. Building modular assets does not mean that the assets won’t have a nice character to them. I wanted to let the asset speak for itself with charm and character. Always good to tell a story!

I created a higher poly wall with cracks and some areas which revealed a wooden frame behind the wall in Zbrush. The walls got baked on planes which gave me the freedom to move the planes on different locations and dress the sides differently for each house. Wooden plank pillars were hiding the open corners of the plane meshes.

My approach for creating those walls was rather fast and experimental. Same goes for the textures used for it, I used some filters to get a painterly oily look, which I ended up liking and decided to keep it. It gives the asset some sort of old but cute feel in my opinion.

The Roof

The roof was a bit more work due to the hand placed and deformed tiles along the surface of the roof. I got 3 mid poly variations of the high poly tiles and decorated the roof with those reduced versions. I was making sure to have enough topology to edit, deform and bend the tiles to fit to the roof mesh.

Then I decided to further “decimate” the whole roof tile arrangement mesh which was formed and hand placed with those mid poly tiles. The lower poly mesh got exported separately and placed on the original roof to get better captures for shadows and light bounces of GI.

At this point, I was missing the variation of the colors of the tiles like my previous texture had but got great results with the rendering of lights and shadows in the engine instead. The arrangement of the mid poly version with around 70k polygon in total got reduced to a 10.5k polygon mesh. Which is totally fine as a compromise.

Now when I think about the coloration of the tiles it is fairly easy and doable by either a mask with rgb channels, masking out different tiles on the texture for the 3 mid poly meshes and then multiplying a solid color on it in the material editor of UE4. Even simpler would be to use vertex color and re-export the asset. I guess I was too busy setting up the renderings to think about such a solution at the time.

One color tint on the roof actually helped out to balance the details of the geometry compared to the rest of the house. Since the geometry was denser than the rest of the house and with some additional color accentuations on top, it might have appeared too important or busy. I did not want to present roof tiles but instead the idea was to create a modular house with some sort of character and charm.

The only problem that I had to solve now was the missing geometry ambient occlusion of all pieces together.

The individual parts look rather thrown together and unnatural without a solution. Especially for stylized art you want to have some ambient occlusion bakes in place. I was using distance fields ambient occlusion and was already happy with the look of it but still wanted to have more control on a texture base. Therefore, I combined all assets for the house together and created a new packed second UV layout. There I was able to create a nice shared ambient occlusion map for a lot of modular mesh parts which are included in the house. I simply hooked it up into the UE4 material editor and it was quite easy to control with some parameters.

Vegetation

I created individual grass blades which I could bend, deform and customize as needed to create a believable arrangement of grass patches and flowers. The meshes for the blades got a gradient node that I put into the diffuse channel of my material. Those will give a great base for the albedo map later on.

A lot of variations can now be created using these grass blades as a base. I combined the meshes together and did arrangements of little grass patches and other vegetation types.

The assets get packed together on a power of 2 plane which will be used to bake down the texture information. From there we can cut out our vegetation cards and start placing them in 3D space.

Ground Vegetation Material and Custom Normals

I found out that there are some important things to take care of when rendering vegetation or grass in UE4. Make sure to use custom normals on the vegetation. Normals can be edited with an edit normals modifier in 3D’s Max. Simply select all grass normals and let them point upwards.

It was really important to get my custom normals working properly by unchecking the checkbox “Tangent Space Normal” In the Material details highlighted in red on the picture below.

I guess I could have used an object space normal map instead. The material itself is really basic, similar to all other materials. I didn’t use all of the roughness or spec maps from Quixel. I was using parameters to get into the right range of reflectance value but more globally, instead of using super detailed microsurface structure for the asset. This gave me a bit more of a plasticy feel to the materials, which in the end I decided to go for. GI bounce lights were way more present this way and bounced off more controlled.

Tree Materials

For the Tree model I decided to use the masked blend mode and subsurface shading model. Two sided rendering was enabled of course.

Dynamic Global Illumination setup with Distance Fields

If you have not enabled distance fields in the project settings under rendering yet this will have to happen now (check the picture below). You have to restart the engine to get the feature

working. Furthermore while Unreal Engine 4 is shut down you have to add a console variable in your .ini file. Locate your .ini file inside the engine directory not the project directory though. Path example: UE_4.15EngineConfigConsoleVariables.ini

Open up the file in a text editor and add following command to it and make sure to save the file and boot up the engine.

– Force no precomputed needs to be checked. Rebuild lights!

– Directional light set to movable and Dynamic Indirect Lighting needs to be checked.

– Post Process Volume set to unbound, enable it and set the intensity high to see the

effect.

Light Propagation Volume (Post Process Volume)

Now comes a fun part and my most favorite feature of Unreal Engine 4. I am the biggest fan of gradients and I think they are the most important tool to our hands for making art appealing. With bounce lights I can create those gradients very easily. As you can see here, I set up a quick test scene for you, so you can see the feature in action.

There are different settings that you will be able to play with in the Light Propagation Volume tab of the Post Process Volume. There you can get the best out of the feature and tweak settings for your needs.

I also added a SkyLight (needs to be set to movable) and an AtmosphericFog to capture the surrounding assets better and bring in more color bounces.

Conclusion

I am really happy how my side project turned out and I will continue to use Unreal Engine 4 for my personal work in future. Epic is doing a great job and it is a blast using such a strong tool to showcase your visions.

My next project is already in the pipeline. Like I said before the rendering of the engine gave me so much tools to get my vision on the screen really fast. Even when the assets weren’t done yet, I already saw how I can nail down my look in the work in progress stage.

Art should be executed without any limitations of tools or any high level technical knowledge. The technology is getting there more and more.

Thanks a lot for reading my breakdown of the stylized rendering scene in Unreal Engine 4 on 80 Level.