Jacob Clode discussed the production of his UE4 stylized scene, Emerald Dungeon, shared his approach to texturing in Substance Painter, talked about the waterfall creation and Maya and working in UE's Niagara system for VFX.
Introduction
Hey everyone, my name is Jacob Clode, and I’m a 3D artist from New Zealand. I graduated from Media Design School in Auckland at the end of 2018 as a Prop and Environment Artist, but I didn't really capture my love for environment art until more recently. My work usually consists of translating the concepts of skilled artists from 2D to 3D. Recreating 2D concepts in 3D is a great way to develop as an artist, and to get a grasp on how these props and environments would work in a real space. Lighting, structure, and composition are things that may need to be changed when transitioning from one medium to another, and knowing how to pass these hurdles is an important skill for 3D artists to develop. I mention this now because it was a hurdle I encountered a lot when working on this piece.
Emerald Dungeon: Getting Started
People often say that you should know your reasons for creating a new portfolio piece before you begin. They aren’t wrong, but it can be hard to predict how your goals may change over the course of the project. I think it's very important to be flexible with your work; your skills and tastes will develop and it's good to be able to adapt when they do. I began this piece wanting a simple environment turnaround but ended up pushing it further and further until the original plan was disregarded. While this made the project take a lot longer than originally planned, it allowed me to express my tastes and flex my skills in ways I wouldn't have originally.
Before starting this project I had been teaching myself ZBrush and Substance Painter, and I knew I wanted to continue working with these software solutions. Since my previous pieces had all been props, I decided I wanted something bigger; an environment. I discussed this with a friend of mine, who began throwing concept after concept at me encouraging me to get started. It took a while to find a concept that fit, as I wanted to find something that I could realistically create as an amateur, but eventually, she showed me a collection of concepts by King Lin that I liked. They were small in scope, interesting lighting and effects, and a style I could work with.
The original plan was to keep it simple; block out the shapes, details in ZBrush, textures in Substance, render in Marmoset, and Bob’s your uncle. With this in mind, I began thinking of how to get started.
Modeling
Prep for the scene was fairly straight forward. The concept was somewhat detailed and small in scope. The scene was able to be divided into four, with four pipes, two bridges, eight doors, six pillars, and what wasn't visible I could ascertain from the scale of the environment. Because of its simple composition, the block-out was quick. In Maya, I divided the mesh into 5 subcategories based on position, which would help for sculpting and when I laid out UVs later down the line.
The first pass was completed in a few hours across 2 days with most of that time being simple scale adjustments, since getting the depth and perspective right was tricky. The majority of the scene was modeled at this point, with the main exception, the angels, being left to be sculpted in ZBrush.
Every aspect of the scene is symmetrical. The centre platform is split in half on the X axis, with 5 obelisks on each side. The back wall is duplicated on its X and Z axis, as its location and lighting mean the mirroring will be less visible in the final scene. This means, despite there being 8 doors in the scene, only two were sculpted in ZBrush. This applies to the majority of the meshes in the scene. Two and a half obelisks, one pipe, one window, two chains, etc. As the original idea was to present the scene in Marmoset and Sketchfab, the original plan was to be very conservative with my UV space, making nearly every single piece modular in some form.
Sculpting
After Maya came ZBrush. Since I didn't want to stray too far from the silhouette of the base mesh, a lot of the work done was fairly conservative. Working using the symmetry axis that I mentioned earlier, I set out to give every part of the mesh some noise. I was going for a rough stone and polished tile feel for the materials, with the metallic aspects being an old, polished metal.
I mostly used Orb’s brush pack to sculpt the stone, as well as some brushes created by Luis Armstrong for the stone cracks. Sculpting consisted of mostly craving out bricks and tiles using Orb cracks, weathering them using some detail brushes, and roughing up the edges with Orb’s flatten edge brush, hpolish, and trim dynamic. From there, it was simply rinse and repeat, with some runes and damage being added for variation and detail.
Retopology and UVs
Since I had created the base mesh in Maya before sculpting, not much retopology was required. Some small adjustments here and there to make sure any major differences between the sculpt and the mesh are accounted for and meshes were altered to match the symmetry of the sculpt. The angels, having been sculpted without a base mesh, were retopologized using Maya’s quad draw tools. The angels all share one basic shape, so only one retopology was required.
During this time I added a handful of extra planes in case I needed them later. Details and lighting effects were created as simple planes to be textured and placed later on, which included planes for fire, smoke, light, water, and webs. Anything I thought I might need, I made room for, probably more than was necessary. This caused a lot of UV adjustment later down the line for one texture set in particular, but at this stage, I was still experimenting with what worked best.
On to UVs, and this was made a lot easier because of my preplanning. I put a lot of thought into efficiency and optimisation, and this is especially the case in my UVs. I had already divided the texture sets based on scene location and knew from the start what parts of the mesh would be symmetrical, so most of the work was simply to cut and layout. I had divided the textures into six sets at this stage; centre platform, bridge, outer platform, details, angels, and planes. The angels were part of a separate set by the time they were retopologized, as the details set was getting crowded by this stage.
Texturing
Next comes texturing Substance Painter, where I begin by baking everything. I want to catch any issues with the UVs or topology early, so I bake each texture set at a low resolution to see if any major changes are required. For baking, I split the high poly into three major chunks (for performance) and simply baked set by set.
As mentioned before, the majority of the scene was composed of three materials; cut stone, polished tiles, and metal details. Figuring out the colour palette was tricky; I didn't want to stray too far from the concept, but it was obvious there was an overuse of grey and the green that was there wasn't vibrant and impactful.
I painted a lot of random colour strokes onto the stones and added red and blue overlays to give some more variation. While obviously, stones don't look like this in reality, it was a quick fix that the darkness of the scene would obscure. A similar change was made to the outer platform tiles and brickwork. The outer platforms won't be as lit as the centre platform, and in darkness, there is very little difference between grey and bluish grey. I decided I had to start pushing colours and textures further. Grey-blue became deep navy, teal, purple, and green, with similar random colour strokes matching the ones I painted on the bridge and centre platform stone.
I remembered following a tutorial by Kalyson a while back (see below) where they make some cracked stone tiles. As part of their substance, they create a nice marble effect by leveling out a simple bubbled tile generator, and I wanted to use that small detail in my scene. It's only two nodes, but it creates a nice webbing pattern that is perfect for adding variation to my stone. This texture became the new base colour for my meshes, replacing the flat grey layer I had previously. While I was at it, I created a tiled texture for the back wall, aiming for the same style and scale as the concept, as a placeholder more than anything. With that, the majority of the texturing was wrapped up, so I moved on to building the scene.
Moving to Unreal
Originally, I had planned on my scene being rendered in Marmoset, and a lot of my decisions had been made with Marmoset and Sketchfab optimisation in mind. Unfortunately, a friend of mine suggested I push the work further and rebuild the scene in Unreal Engine instead. So I did.
This caused problems.
It had been a long time since I had worked in UE4, so I had to spend a lot of time relearning everything. Unreal renders materials differently than both Substance Painter and Marmoset, so once I committed to the switch, a lot of work had to be edited. Emissive and metallic maps had to be made weaker, but normals and roughness maps had to be made stronger. It was a process of trial and error, making slight changes to get the results I wanted.
Originally, the difference in presentation was frustrating, but over time I began to like the new style the scene began to develop. The biggest issue was transparency; when first implemented the transparent glass texture was part of one texture set, alongside the pillars and doorways.
Substance (and Marmoset) allows for texture sets to have both transparent and opaque areas without it causing any issue, so I had UVs laid out by location in the scene, rather than by material.
Unfortunately, I wasn't aware that Unreal didn't render its materials the same way, so this had to be changed. I split the transparent attributes of each texture set (the glass, the smoke, the lights) into a new material, as well as updating my meticulously laid UVs to work better in the new engine. Sometimes the best fix is the simplest.
Animating the Scene
The move to UE4 had the benefit of allowing me to add more animated aspects to my scene, such as particles, advanced materials, and tech art. I started small; I wanted to add rotation to the beam of light at the centre of the scene. I went back into Maya and broke the layers into individual cylinders, and added them back into UE as an actor. Each piece rotates on the Z axis at a different speed, with very slight Z translation for variety. This was a very simple effect, just to see if committing to animating the scene was worth it; it was.
Following this, I binned a lot of the planes I had made. They were originally used to simulate VFX and the change to real-time meant they were no longer necessary. Light beams, waterfalls, bubbles, smoke planes, and candle planes were all removed and replaced. Most of the replacements were simple; the ‘light’ emitting from the doorways is a simple transparent texture, overlaid and panning in opposite directions. However, some were much more complicated than the original idea.
Tech Art
The biggest new addition to the scene is the four dynamic waterfalls under the platform. Originally, these were simple panning textures, but it became obvious that the bottom half of the scene was really lacking compared to the top. I needed something colourful and obvious to catch people's attention.
I remembered seeing a blog post by Math Roodhuizen a while back in which he creates an awesome waterfall based on Rime. I wasn't able to figure it out at the time, but I decided it would be an excellent fit for the project so I got to work creating a similar effect in Unreal.
You can find Math’s full breakdown here if you want to learn more about the specific techniques and details.
The ripples effect is really simple; two identical RGB ripples are panning at different speeds in slightly different directions. The first thing we’ll need is a flat mesh designed to pan outwards. I made this by flattening a cylinder and laying the UVs edge-to-edge. I added a few edge loops to the plane and went about painting vertex colours onto the mesh, which will be used later.
Onto the UVs. Since we are using a tiling material, the UVs must be scaled correctly. The waterfall is a long mesh, and to prevent texture stretch, we want the UVs to scale 3-4 times on the V axis. This is based on the scale of the mesh; the smaller the mesh, the less it needs to scale, but the scale has to be uniform. Since we’re starting at 0, my UVs are 4-1, 3-1, and 2-1, for the three waterfall tubes. U axis scale is less important; since the seam will be hidden at the back of the mesh, it doesn't need to tile perfectly.
We also want to have some variation in the movement speed of the texture, so we’re scaling the UVs to be tighter in certain areas. Take the vertices that make up the bend, and using the soft select tighten them together. Same thing, on the main body of the mesh, spread the vertices slightly apart. We want the water flowing fastest as it goes over the edge, and a bit slower as it is flowing freely. This isn't an exact science so you can adjust accordingly to your needs, just make sure your mesh is preserving its shape since we’re only wanting the UVs changed.
Creating the Materials
With the mesh setup, we move on to the materials.
Again, the material itself is fairly straightforward, using two simple textures to get nearly all of the results. I started with a panning water normal texture with parameters for scale and speed on the X and Y axis. The waterfall itself will become very choppy as we start to use displacement, but the normals add a nice variation to the areas that aren't moving as much.
Next comes the opacity mask, which is how we get the nice holes and ripples throughout the waterfall. This follows similar rules as the normals; textures panning at different speeds and scales, but this time we’re using a Perlin noise texture. Duplicate into two, adjust the speed and scale of each, and take the red value of one (R), and the green value of the other (G).
Multiply both these values with some parameters, and add them together, before finally multiplying with a vertex colour node (with a 1-x node in between). Punch the result into your opacity mask and voila, you get your holes and ripples.
The displacement uses the same Perlin noise textures and vertex colour node as the opacity mask, but the panners are added together before being lerped and multiplied by a world space vertex normal node. Plug that into your world position offset and the displacement will be ready.
VFX
Since the last time I had used UE4 Epic had replaced the old Cascade system I was familiar with the new Niagara particle system. Niagara is a really powerful and practical tool, but it wasn't one I had any experience with. I tried to keep a lot of these new effects very simple, such as popping bubbles and flickering fire and decided to place them throughout the scene so no one section was more distracting than the rest.
The effect that got the most use was a simple smoke emitter with some basic attributes; low opacity, slow rotation, a lifetime of 2-10 seconds, collision, and gravity. From there steam was added to different parts of the scene, with variations depending on the use case. The steam coming from the doorway was dense, slow-moving, and very transparent. The door steam was a thick white fog with a long lifetime that slowed and faded as it nears its death. This steam has nearly no vertical motion, instead rolling across the mesh until it falls off the edges of the bridge. The steam emitted by the water pipes was much different; tightly packed emitter, lime green, fast movement, and a short lifetime of under five seconds. This steam floats out in every direction but has a strong vertical velocity, and is less dense than the others so as to not obscure the flowing water.
Overall, I used the master emitter in five different Niagara systems, from the steaming pool to the candle smoke. A lot of the effects were cut, such as smoke building inside the lamps, or bubbles floating away from the ripples, but these were all removed for either adding too much or too little. Sometimes simple effects are all you need to add to the mood of an environment.
Lighting and Details
The final chunk of work (and possibly the most difficult) was the lighting. Unlike mesh work or texturing, which allows for a lot of creative freedom and creativity, I find lighting to be a difficult balance of style versus realism. Since the scene was originally planned to be rendered in Marmoset, a lot of colours were more desaturated and grey, and the lighting set-up was very basic. I realised pretty early that I needed to push the colour palette a lot further once I was in Unreal, as the entire scene was significantly darker than it was at the texturing stage.
When I began lighting the scene, I was following the concept pretty closely. However, it was pointed out to me that the way I was lighting the scene did not accurately represent how lights worked in real life. Some lights had bloom, others did not. Light radiated from sources that didn't affect the environment around it. The most obvious difference was the centre platform; in concept, the platform has a spot of moonlight coming in from what must be a small hole in the ceiling. However, my scene had no such hole, and it didn't make sense for the light to be as sharp and bright as it was.
I got some helpful critique from a friend who threw in some extra lights and overall made the scene look a lot more natural. The first change was the move from baked to dynamic lighting; I wanted the lights to flicker and move, and I didn't need to save on performance. Since the lights weren't affecting the scene how I wanted, I increased the global illumination and made up for the change by increasing shadow intensity in post-processing. I disabled shadows on a lot of the lights, as they simply were not behaving the way I wanted. Next, I threw in plenty of reflection captures and bumped up the reflectivity of the textures quite a lot, which gave the scene a lot more depth.
Conclusion
Normally, the biggest challenge I face when working is staying enthusiastic about the piece. My work can often take months, and as it progresses it can be difficult to find the drive to finish everything. This piece was no different. I was about 80% of the way through the original plan when it was suggested I should switch to Unreal, and when I did that progress went down significantly. There were a lot of changes that had to be made, new things to learn and to worry about. Overall, I'm not sure if it's a situation that I would suggest anyone put themselves in, but I'm glad that it happened because it made for an interesting challenge.
In the past, if someone wanted to try something new, or saw faults in their work, I’d have suggested they move on and start something new, but I'm now encouraging people to try something new for the fun of it. While losing so much progress was demotivating in some ways, it was motivating in far more. I had an opportunity to do lighting, tech art, VFX, animations, art that I otherwise might not have been confident to put into my work, and I was much better off for it. If you think you can make something better, do it. If something intimidates you, or you’re worried you can't get the results you want, try anyway. Don't be afraid of doing something wrong, or doing something different. While it's important to plan ahead and know what you're wanting to accomplish (especially for portfolio pieces) never ignore a chance to learn when it's in front of you.
Thanks for reading my rambles.