First Encounter with Unreal Engine Turned into Beautiful Swamp Diorama

Introduction

Greetings, I'm Anuj Kansara, a 33-year-old individual hailing from Mumbai, India. Currently, I'm based in Bangalore, where I serve as a senior environment artist. My journey into the professional realm commenced towards the end of 2014. Initially, I delved into the realm of simulation games, contributing as an environment/props artist; these games were tailored to train a diverse range of working professionals. Subsequently, I transitioned to working on indie VR titles tailored for the Oculus Go platform. This endeavor afforded me the priceless chance to explore art direction and various facets of gameplay design. 

My passion for creation dates back to my childhood days spent tinkering with building blocks and LEGO sets, instilling in me a love for crafting. Later on, I was introduced to the world of gaming, and I found myself deeply captivated by the process of their creation, particularly those rendered in 3D. It was during my college years that I started exploring software such as 3ds Max and Photoshop, solidifying my desire to pursue a career in 3D art. 

I pursued a diploma in animation and VFX at Frameboxx Pvt. Ltd., Mumbai, graduating in 2011. Since then, I have remained dedicated to continuous learning, constantly exploring new tools and techniques to stay at the forefront of the industry. This dedication paved the way for my entry into the professional arena in 2014, marking the beginning of an exciting journey as an environment artist.

Unreal Engine

After having spent considerable time on Unity projects, I desired an opportunity to explore Unreal Engine due to its renowned cinematic aesthetics. Despite facing numerous challenges that didn’t allow me to dedicate time to learning it, I was determined to give it a try for this particular project. One of the driving factors behind my decision was the variety of tools, plugins, and assets available at the Unreal Marketplace. Plus, Lumen and Nanite made things faster and better. Additionally, the abundance of tutorials created by skilled artists made the learning process much more manageable and rewarding.

Swamp Diorama

I stumbled upon miniature dioramas through a friend, and ever since I couldn't shake off the idea of recreating one in 3D. What fascinated me about dioramas was their compactness, which provided the perfect canvas for crafting detailed assets within a limited area. With this in mind, I decided to base my scene on a concept that I had come across on Google. 

As a big Counter-Strike fan, I always wanted to recreate some of the game's maps in 3D. Although I never got the chance, I combined this aspiration with my interest in dioramas to create the concept of a terrorist hideout. Later when Valve dropped the Operation Riptide teaser, featuring the reintroduction of an old character skin, Guerrilla Warfare, which was set in a jungle. This sparked the idea of a group of terrorists seeking refuge in an old hut sheltered within the swampy woods. Here, they engage in activities such as stockpiling arms and ammunition, crafting explosives, and planning a terror attack on a nearby target.

Once I finalized the layout of the scene, I delved into researching and gathering props that would complement the composition. I carefully selected props from both CS:GO and a few from Half-Life to ensure the scene felt complete. Taking inspiration from games such as Hunt Showdown and Red Dead Redemption, I aimed to recreate their atmospheric environments and lighting effects found in various weather conditions. Setting my sights on achieving textures and details comparable to the quality seen in Metro Exodus, which I consider top-notch, I strived to emulate their standard. However, I'm still uncertain if I managed to match it. As for the boat, I drew inspiration directly from Metro for its detailed texturing.

Here are the references I gathered for the main hut, terrain surface, and other organic assets for the scene.

This phase was critical as I focused on selecting props to convey the storytelling elements effectively. As a result, the number of props increased to around 80, ranging from small to large, the challenging aspect was ensuring consistent quality across all of them.

Here are some in-game screenshots I captured to better visualize set dressing and replicate mood and scene composition.

Blockout

Once I had a clear understanding of the structure and terrain references as well as the layout of the scene, I began by blocking out a basic low-poly mesh of the hut using 3ds Max. Similarly, I created a simple ground mesh to approximate the size of the area surrounding the hut. The primary goal at this stage was to test out the proportions of the structure, determine the size of the terrain, and familiarize myself with the basics of Unreal Engine.

This was my initial export and my first venture into Unreal, where I experimented with the 3ds Max to UE workflow and familiarized myself with various tools, materials, and the default lighting setup. I utilized Megascans materials on the basic ground and painted some grass along with a few 3D assets from Megascans, serving as placeholders.

At this stage, I had become more familiar with Unreal Engine's workings. Consequently, I began experimenting with foliage assets that I intended to incorporate into the scene, while also conducting some lighting experiments. I tackled these tasks together during my short breaks from modeling or in my limited free time during the day.

Here's a quick collection of screenshots taken from the 3ds Max viewport during the modeling process.

At this stage, I had finalized the terrain, structure placement, and tree arrangements. I selected the foliage types from Megascans that best suited the scene, created a temporary RGB-masked material for the terrain, and incorporated Megascans forest ground textures. I also tested fallen leaves and forest debris to see if they would work well in the scene. 

Goals

Before exploring the workflow, I'd like to share some insights into my project vision and its ultimate objectives. These considerations heavily influenced the workflow I devised for the project.

1. Since this was my first time working with Unreal Engine, my goal was to learn the environment art workflow according to industry standards and to familiarize myself with the tools used in creating a game-ready environment.
2. I wanted to create a detailed and compact artwork filled with assets that tell a compelling story. Ultimately, I aimed to produce a cinematic video with a gaming vibe.
3. I aimed to enhance my texturing skills and elevate my ability to achieve realistic details by mastering the creation of procedural textures in Substance 3D Painter.
4. I'm passionate about sculpting and wanted to improve my skills, so I planned to devote a considerable amount of time to ZBrush to craft highly detailed assets. By doing this, I aimed to take full advantage of Nanite's capabilities in Unreal Engine.

Modeling, Texturing, and Shader Setup

Terrain

As illustrated in the GIF, I approached the terrain work methodically. Initially, I prepared a basic ground mesh in 3ds Max, then enhanced it in ZBrush by adding undulations and refining it to more closely resemble natural terrain. I also sculpted areas for water and, using the structure's block out, created a final terrain mesh. After completing the mesh, I experimented with different workflows for the terrain material and decided on the most suitable approach for my needs.

Subsequently, I manually placed rocks and tree debris and used foliage painting to add smaller stones. Being relatively new to Unreal shaders, I turned to a helpful tutorial on ArtStation Learning by Dekogon, which offered valuable insights into creating hero props and game-ready environments. I also followed Stef Velzeboer’s tutorial for creating a water shader tailored to my specifications.

With the positions of structures and trees already set, I focused on the foliage painting, for small plants and used European Black Alder Trees from Megascans to complete the terrain.

After exploring various tutorials, I selected a workflow and crafted an RGB-masked material for my terrain, given its compact size. I divided the terrain into two sections, upper and lower, and applied two distinct shaders to them. Using Substance 3D Painter, I created the RGB masks and incorporated forest textures from Megascans for the terrain. For the lower section, I added roots and dirt textures to distinguish it, using two common textures in both the upper and lower sections to blend the meshes effectively.

Additionally, I integrated a wetness mask in the upper terrain where it submerges beneath the water to enhance realism. To ensure a natural blend of the assets that contact the terrain, such as rocks, stones, and tree debris, I utilized Unreal Engine's Runtime Virtual Texturing feature, which allowed for seamless blending of assets into the terrain.

While developing the shader, I kept modularity in mind, aiming for a master material that could be reused in future projects. I incorporated multiple features into the terrain shader, planning to add more in the future as I keep learning and to improve the shader's functionality and ease of use.

I organized the terrain props separately from other props and adhered to the standard high-poly to low-poly workflow. For the rocks, I sculpted them from scratch in ZBrush. For the tree debris, I initially crafted a basic low-poly mesh in 3ds Max and then added finer details in ZBrush. For the low-poly versions, I lowered the mesh polycount using ZBrush's decimation tool, although I didn't focus much on reducing the polygon count since I planned to use these assets with Nanite, and later unwrapped the assets in 3ds Max.

All texturing work was done in Substance 3D Painter. I created a base smart material for both the rocks and tree debris, from which I made additional variations.

These are the assets specifically designed for the terrain. To enhance their variety, I used a few shaders with different tints on them.

In the water shader tutorial provided by Dekogon, they also explained how to implement a wetness function for assets that will be submerged, to enhance realism. This guidance simplified the process for me, so I adopted the same approach for adding a wetness function to the props. This function leverages the world position to create a mask where everything below zero or a specified level alters its roughness to zero, achieving a realistic wet appearance.

When I began placing the terrain props, they looked quite out of place because there was no blending where the assets met the ground. Unsure of how to resolve this, I started researching potential solutions and discovered that Unreal Engine offers a fantastic feature called Runtime Virtual Textures. 

This allows for dynamic blending of multiple textures on a landscape based on parameters such as height or slope. After studying a few tutorials, I was able to create a material function that effectively addressed the blending issue.

Before beginning work on the materials, I watched several tutorials that demonstrated the dynamic use of moss, giving me a clear idea of how to add moss to the terrain props. I then came across an excellent tutorial created by Lukas Koelz that featured a material function with world-aligned moss, which adheres to the top surfaces of the props and includes various other features.

This master material has been specifically developed for all terrain props, incorporating multiple material functions to ensure the assets blend seamlessly with the terrain.

Props and Structure

Working on props was one of the most crucial tasks as I had decided to treat all of them like hero props and wanted to maintain that consistency across all of the props also managing the time it took to complete. I followed the high-poly to low-poly pipeline, I quickly created a basic low-poly model in 3ds Max, sculpted all the details in ZBrush, and used Topogun to retopologize all the high poly props as it gave me a lot of freedom.

I also experimented with the remeshing and decimation workflow, but I wasn't quite satisfied with the results. Therefore, I decided to stick with the high-poly to low-poly pipeline and also took the opportunity to try my hand at using Topogun.

All the texturing for the props was completed in Substance 3D Painter, where I created a variety of smart materials using generators, filters, and custom masks. I relied heavily on procedural texturing techniques to achieve detailed and consistent results across the props.

To reduce repetition among similar types of props, I added a function to the material that overlays a dirt texture onto the existing texture. This can be adjusted to create variations in dirt, primarily to achieve the effect of ground dirt where objects come in contact with a surface.

These are all the unique props designed specifically for this scene.

For the structure, I initially planned to create a modular design with reusable wooden planks and utilize vertex painting for texture variations. However, since there was only one small structure, I decided to make unique assets and sculpted the entire structure in ZBrush, reusing just a few wooden planks. After sculpting, I decimated the structure and utilized them as Nanite assets in the scene.

For texturing, the structure was divided into parts, each receiving unique textures. I developed a smart material in Substance 3D Painter for the structure that included overlays of dirt and moss, applying it throughout with various adjustments to ensure there was no repetition. I applied the same material workflow to the structure that I used for the props.

These are all the unique assets created for the structure.

Decals

These are the unique decals created for the scene, which were designed in Substance 3D Painter. Decals are crucial for breaking up repetition and adding realistic details such as stains and weathering effects to surfaces. Adding decals was a subtle yet effective way to enhance the storytelling within the scene.

Other Assets

Cloth

I used the cloth modifier in 3ds Max to simulate cloth and then I implemented cloth physics in Unreal Engine to add realistic dynamics to the fabric in the scene.

Rope and Chain

Initially, I tried using the Spline blueprint, but it didn't work out as expected, so I reverted to the traditional method. I created a modular piece and used the Path Deform modifier for the ropes and chains, manually stitching the rope ends, and used FFDs (Free-Form Deformations) for variations. Later, I placed the ropes manually in the scene and utilized the lattice tool from Unreal Engine's modeling tools to make adjustments as needed.

Particles

The scene with the particle effect:

The scene without the particle effect:

Additionally, to make the scene more vibrant and lifelike, I added several particle effects. I utilized some of Unreal Engine's starter content particles, including Ambient Dust, Fire, and Smoke, and also created my own, such as mist particles and ground fog, to add more depth. I followed multiple tutorials to achieve the results.

One especially useful tutorial was from Flipside 3D on creating dynamic fog materials, which played a key role in improving the atmospheric elements of the scene.

For the stove/sigdi, I utilized Unreal Engine's default fire and smoke particles, tweaking the settings to suit my specific needs.

I created the moth particles using vertex offset animation and animated the world position of a single texture for the lantern flame. Check out this tutorial I used for creating moth particles by the talented artist Pierrick Picaut.

Lighting

I wasn't very familiar with lighting techniques, so I didn't have a detailed plan at the start. Throughout the scene creation process, I spent time experimenting with Unreal Engine's lighting features and watching tutorials to improve my skills. Eventually, I discovered a very useful plugin called Ultra Dynamic Sky, which greatly simplified many aspects of lighting for me.

This plugin was particularly helpful because my goal was to quickly create different weather conditions. For all the daytime scenes, I used just one directional light source and a skylight, along with different dynamic cloud settings. 

For the cinematic video, I aimed to showcase a day-to-night transition, starting with sunny weather that gradually shifts to rain. The UDS system was ideal for this, as it's perfectly designed for dynamic weather scenarios and user-friendly for beginners. I experimented with various settings, including camera angles, cloud density, and fog levels, to achieve a seamless transition from sunny to cloudy conditions.

This showcases the flexibility to easily change and control the weather settings. Additionally, the plugin includes a material function for wetness that can be integrated into your own shaders. It offers extensive options for adjusting the wetness level on your assets and controlling how it changes over time.

For the night scenes, I added an extra directional light with no shadows with very low intensity and spotlights for the lanterns, enhancing the mood and detail of the scene. I controlled the cloud movement and animated the time of day, which included both daytime and nighttime, to effectively assist in the time transitions for the cinematic video. I maintained a darker ambiance and primarily depended on manual camera exposure settings and further enhanced the output while doing the post-processing in DaVinci Resolve. 

Rendering and Post-Processing

Here are the render settings I implemented. While I explored numerous tutorials, I specifically adhered to William Faucher's guidelines for rendering and post-processing, as his settings aligned perfectly with my project requirements and yielded the best output. Additionally, I incorporated some extra console variables discovered online. Although I'm uncertain if these had any impact on the final quality, they did not extend the render times, so their inclusion was risk-free, and it gave me peace of mind.

For the final output, I rendered the footage 4K/24 FPS in EXR format and employed the ACES workflow for the post-production in DaVinci Resolve. 

Here are some before and after screenshots for comparison.

While I'm not formally trained in color grading, I thoroughly enjoy experimenting with the final output, which led me to establish a solid workflow using DaVinci Resolve. I found it more comfortable and user-friendly for adding effects and making tweaks compared to Unreal Engine's post-processing tools. By playing around with a few nodes, I significantly enhanced my output.

I made adjustments using the color wheels, applied some color corrections, and added film grains, a bit of sharpening and glow effects, and lens flare for the still images. All these enhancements made a subtle difference in the final output.

All my thanks go to William Faucher, whose guidance was invaluable in rendering and post-processing.

Challenges and Lessons

I was confident in the workflow since I followed the traditional method, which I am very familiar with, so I didn't encounter many issues with asset creation. However, some automation tools, like decimation and remeshing, did not perform as expected, which made time management challenging. Retopologizing the numerous hard-surface props became particularly time-consuming due to their high count.

Having experience with game engines, I found picking up the basics of Unreal Engine (UE) relatively straightforward, even though it appeared intimidating at first. However, rendering presented a significant challenge because I was unfamiliar with UE's specifics. Thankfully, numerous online guides helped me overcome many minor issues. 

A major problem I faced was shadow flickering during video rendering. Initially uncertain about the causes, I consulted a wide range of online resources, which helped me adjust my render settings effectively and resolve the issue. While rendering stills in Unreal is generally simple, producing videos requires multiple adjustments and attempts to achieve optimal results.

I've grown significantly and learned a great deal through this project. Balancing professional and personal life, maintaining work-life equilibrium, and handling challenges patiently have been integral parts of this journey. I also want to extend a big thank you to all the talented artists out there who work hard to share their knowledge and make learning more accessible for everyone.

I am currently planning to create a larger forest scene using the same assets and structural setup but with varied weather and lighting conditions. My goal is to develop a playable build that I can upload online, making it accessible for people to download and explore the environment.

Tips for Artists

Here are a few tips based on my personal experience with this project:

• Determine the art style, workflow, and pipeline. Decide how much detail you want in your scene. For highly detailed scenes, keep them small and simple. If you're aiming for a larger scene, set the camera angles first and concentrate on adding details to key areas. Use a modular pipeline and try to reuse as many assets as possible, all while considering how much time you're willing to invest in your project.
• Start with something manageable, like a single prop, and keep creating and learning simultaneously. Continue to develop assets for the same scene. As you dedicate more time, your skills in texturing will improve. Over time, compile these assets to build a complete scene. Texturing can seem like magic; the more time you invest in it, the better you become. Remember to be patient – the results will come as you continue to refine your skills and techniques.
• For the final results, take a few renders and set them aside. Review these renders after a day or two; you'll likely have a fresher perspective and be better able to judge and make necessary adjustments. Continuously learn from others – the community and online resources are incredibly valuable. Engage with the work of other artists and integrate what you learn into your own practices. Remember to take regular breaks and stay hydrated.

I hope this is helpful, thank you for your time.

Anuj Kansara, Senior Environment Artist

Interview conducted by Terri Mills