Lighting a Gloomy Venice-Inspired Scene in Unreal Engine

Venice: Calle Del Riccio

The project Venice: Calle Del Riccio started as a school project. We could choose a city for a building reference and a game as a style reference. I didn't want to pick a combination that already existed, like Assassin's Creed in Ancient Greece. I chose Venice as a city and The Order: 1886 as a style reference. I believed those two could be an interesting combination where early dawn Venice meets the aesthetics of the atmospheric, moody Order. The original assignment was all about creating a modular kit. I thought Venice was a nice middle ground of difficulty, with a huge variety of buildings and details, yet containing clear reusable parts.

The assignment from school was all about a proper modular kit, so I decided early on to completely focus on the building itself, which meant no composition and no set dressing at all. I solely focused on the kit, the wall materials, etc.

I proceeded with the project after a year, because I aimed to create an impressive portfolio piece, showcasing everything I wanted to prove on my portfolio before applying for an internship. The work I did on the Venice modular kit during the school project was a solid base for constructing the cohesive scene.

I looked for nice compositions in paintings, photos, and reference images I took while rowing through the canals of Venice on Google Maps. I picked a few interesting facades and drew a little top-down sketch of the layout of my building.

The Modeling Process

The Modular Kit 

The beginning of the project was all about setting up a proper modular kit. I took my time to look at a ton of references from Google Maps. Venice had very similar buildings, which yet varied in many ways. I wanted to capture a stereotypical Venice building in one kit, without the repetition shining through too much. I looked for common elements between buildings. I started sketching possible modular pieces on screenshots to see how big a modular piece could be and how I could have multiple versions of the same measurements to swap between them. For example, on the right building, the wall space is quite smaller than the window pieces, but I decided to make them the same width so I could swap these pieces around. 

What you see above is the result of brainstorming in Photoshop, and it doesn't reflect the actual final pieces.

Doing extra research on how other games set up their modularity really helped me as well. I found out that Assassin's Creed II took place in Venice. Even though the game was quite old, it was still useful to see how they had recreated the whole city. The thing which actually stood out to me was that I saw way more horizontal trims on buildings in the game than in real life. However, despite that fact, it all worked pretty well, leading me to subsequently implement it in a way that allowed me to horizontally tile my materials on the modular pieces.

Unique Assets

About a year after I had finished the school project, I decided to make a whole scene out of it. The kit was working nicely, but the modular kit alone wasn't enough for a complete scene. I needed some unique assets to break up the repetition. Hero assets should grab the attention, diverting the viewer's focus from any repetitive elements in the kit. To achieve this, assets like the flag, the lanterns, the bridge, and the well were thoroughly sculpted and uniquely textured.

I also made small assets that were scattered more often, such as the fences, the chairs, the tables, and the laundry. Additionally, I created more simple elements like the metal pieces on the wall to keep cracks together. These pieces were greatly hand-placed to conceal more noticeable repetitions, particularly when combined with the crack decals I made.

I probably should have turned the ornamental details into a trim sheet, but, for this project, I simply sculpted the details on the lantern itself.

Here you can see the cloth simulation that was created in ZBrush.

Laundry


The laundry was an interesting one. I made a spline tool for my drainpipes with the help of Ryan Laley's tutorial on spline meshes, which basically instanced the same mesh after a certain amount of distance on the spline. I thought I could rework this tool to spawn a random piece of cloth with a random material instance. I thought through the concept and got the basics to work pretty fast. However, there were many issues occurring, for example, I got completely different generated outcomes every time I opened up the scene. It took me a long time to fix those issues. Eventually, I decided to turn the generated blueprints into a static mesh. This resulted in a new mesh, new draw calls, etc. Basically, a completely new mesh, losing all of the benefits of the tool. I decided to export the splines to Maya and composite the laundry lines there. As a result, I got better outcomes with significantly less inconvenience.

My lesson from this is that it's fantastic to think of the tools and see if you can make them work to speed up your process. However, sometimes you just need to cut your losses, when it's not working out and simply do it the old way. This might still save you time. 


Foliage


The foliage was split into two sections: flowers and greens. 
The greens were all made in the Substance 3D Designer software, while small plants were created during the school project. It was a question, whether it was already part of the basic building, or it was set dressing. I felt it was important to already have it since the greens really helped with the grounding of the building. The plants emerging from the bottom stones and the ivy cascading around the structure greatly enhanced the overall appeal of the modular kit. 


The flowers were added nearly at the end of the whole project, but it definitely was worth the effort. Due to the tight schedule, I decided to create an atlas using cut-outs from Google Images for the base color. I assembled them to work nicely together for bigger assemblies with nice silhouettes.

The Texturing Workflow

Material blending

The most important part of avoiding repetition with such a small kit was the material blending, which was time-consuming. I started by making the three basic materials I needed: the bricks, the plaster layer, and the colored top layer, and then blended them with vertex painting in Unreal Engine.
The images below show the progress of the material blending during the first weeks. 

I made significant progress going from blockout blending to somewhat decent, and eventually, to somewhat actually pretty nice. Initially, my focus was on ensuring that the blending wouldn't be a straight gradient. I achieved this by incorporating a Height blend, using the Height Map of the underlying material. That's exactly what I did for the bricks-to-mortar blend. This approach ensured that the mortar would start between the bricks and in the cavities, only becoming visible on top of the bricks, when the value became high.

For the mortar-to-top plaster blend, I utilized a slightly altered version of the Height Map. Firstly, I inverted it to ensure that the top layer would appear first on the higher parts of the mortar. Additionally, I quantized it to avoid a pronounced gradient, instead giving the parts with the most height difference in the original map an advantage in the material blend. In the material blend, I incorporated a subtle use of the Normal mortar material. In the blue channel, I applied a color overlay, allowing me to paint the plaster color onto the bricks slightly. This adjustment significantly enhanced the cohesion of the materials and improved their overall appearance.

Most of the time, during this phase, went into simulating depth in the blend. I wasn't sure how to accomplish it, but I needed it. I didn't know how to add an actual Normal to the blend yet, so I came up with a trick that gave the illusion of depth.
 I took the gradient outputs from the vertex channels and subtracted the high-contrasted version (the one used for the higher layer) from a lower-contrasted version. That gave me a mask of a sharp line, where the materials changed, and a soft transition on the other side. I used that mask to overlay a highlight on the top layer and Ambient Occlusion on the lower layer. Now I had the illusion of depth.

Polishing the Material Blending

Later, when I got back to the project, I went through quite a few more iterations on the material blending.

The grunge is the biggest difference between the first and second pictures. That was an important part of the walls I still missed. For a long time, I wasn't sure how I would tackle this. I could put it in my blue channel since that was only used for painting in the color tint on bricks, which I could simply have standardized with a decent gradient length. Maybe, I could even use decals? Then I remembered there was also an alpha channel in the vertex colors, so I decided to use that one.

I used a greyscale grunge texture and applied it in a world-aligned manner to the walls. By doing so, I achieved variation across all the wall pieces. I used grunge with verticality in it, so when I blended it in, it looked like streaks going down. While painting the texture, I paid attention to where leaks would go. I specifically focused on applying them underneath trims, as well as below windows and balconies. 

As you may have noticed, I also lowered the water level at this point. I added the first pass of water damage to the wall material between the second and third images. At this point, the biggest issue was the texture limit Unreal Engine had for the material. I wanted to incorporate the new material to blend in with some nice algae and barnacles, but I couldn’t do that because of the limitation. Even though I did stack most of my textures, I didn't do it perfectly, and I had to separate height blends instead of only the height maps. At this point, I decided to simply use world-aligned grunge masks as my water damage. Basically, the same as my grunge, however this time I didn't have a vertex channel to paint it with. So, this time I used a gradient based on world position. Everything below a certain Y got the water damage applied. This was a nice and quick way to ground the walls in the world. I used different gradients for my roughness and base color water damage, so the highest part of the building would be dried up and near the water, it would still be completely wet.



The most significant distinction between the third and fourth images is a crucial one. I added an actual normal map into my material blend, thanks to the material transition nodes introduced by Kyle Steward. It translated your vertex-painted mask into a height difference which got a Normal Map applied. That gave my walls an extra little pop of depth.
 In this stage, I also increased the Normal of the bricks on the mortar layer to blend those a bit better. Particularly near the top plaster layer, the bricks are no longer visible through the mortar.

Trim Sheets


All the stones and wood in the scene were textured with trim sheets. Those materials were also completely made in Substance 3D Designer. I wanted to challenge myself by creating the ornaments in that software as well.

I made the trim using a way discovered by my friend Máté Válent. He used a text node combined with a circular splatter to generate tile patterns for his As Favelas project. I decided to experiment with a variation of it and crafted the trim by mirroring a text node. By inserting a transform node in between, I could easily type something into the text node, observe the output of the mirror while transforming the text, and identify visually appealing results. That worked perfectly. 

I made two versions of the wood to paint with vertex colors in Unreal Engine. One regular and one painted/wet version. I also used the acanthus leaf I made for the stone trim on the wood trim, but I made it look like it was carved. 
Since I didn't have a lot of metal, I decided to add metal trims to my stone sheet. All the metal in the scene, except for the fences and lanterns, was laid out on these trims. All the stones you see in the scene were also unwrapped on top of this. I also ensured that the gutters and window frames were UV-mapped to fit the trims.

After the 8 weeks of the school project, I had two buildings, which were a basic one and a more complex building made later. Since I made sure every angle was different for the buildings, I decided to simply duplicate these two buildings around to create a basic composition. I went through my reference again and looked more specifically into paintings of Venice. I didn't intend to copy any of them, but it would be a nice guideline for general shapes. If you do this, be careful that you keep in mind what makes the painting work. If you only use it as a guideline, you could potentially remove the aspects that made the composition work (the values, colors, shapes, etc.).

Eventually, I decided on a basic composition for my main shot. While working on the scene, I frequently flew around and soon established additional camera angles. I dressed the scene as if it were a game environment, but I focused my effort on these few angles only. I had initially wanted to showcase in a video that you could walk on the bridge and have a nice overview of the buildings along the canal. However, doing so would have required me to dress the entire scene in a different way as well. To save a significant amount of work, I made the decision to refrain from rotating my camera in a different direction.

Lighting


During the school assignment, I was trying to aim for the mood of The Order: 1886. I looked at a ton of references from the game and how they tackled lighting. I didn't completely follow their pipeline, but it was still useful to know. I decided I wanted to do Venice on an overcast day during the fall. I thought this would mix really well with the Order mood. It would allow for a ton of moody mist/fog on the water.
 As I progressed through the project, I kept iterating on the main lighting and post-processing settings which would replicate those of The Order: 1886, like the film grain. The aspect ratio was also chosen to match the one of The Order. At the end of the school assignment, I ended up with a decent base to portray the right mood.

When I continued with the project and I got closer to the final composition, I started tweaking the lighting again and added a bunch of extra lights, mostly for nice rims. The goal wasn't to stick as close as possible to The Order anymore. I wanted to be free and explore whatever worked for the scene, not limiting myself to such a constraint helped with speed as well. I used a fog card behind objects which I lit with a rim light pretty often to accentuate the depth of the scene. This technique effectively made the objects stand out and added visual impact.

Here I didn't use any extra lights:

By applying the additional lights, I aimed to enhance the emphasis on rims and details, subsequently accentuating the elements I wanted to stand out even more.

I knew that I would probably want a daytime scenario as well. I started working on it when my scene was close to its final phase. Periodically, I took screenshots that were twice the screen size to check my progress and gather feedback. Combining all these images in a PureRef-like display, as shown below, greatly aided in visualizing the progression and confirming that I was making progress. This technique was immensely helpful in maintaining motivation during the polishing phase, where a lot of work was done but visible differences were minimal.

Since I'm quite a bit colorblind, it's extremely important for me to ask for regular feedback on my colors. Luckily, I have plenty of amazing artists around me, to whom I can send constant updates on my project. Máté Válent, in particular, helped me with the final color grading on my daytime scene as well as constant feedback throughout the whole project. It is important to have a solid base without any color grading or post-processing, but adding it near the end is essential to give your scene that final push. 

Final Advice

The school project lasted eight weeks. During this time, I did most of the research and I completely dedicated myself to the modular kit because I knew I wanted to dive deeper into it. A year later, I spent another two-three months of evenings and some weekend days to make the rest of the scene.



I believe that keeping motivation and dedication at a high level is crucial. Part of this was because I didn't see it as just a job. It was difficult at times when I had to explain to family and friends that I was taking "coloring lessons," and it made me feel like I was back in kindergarten. Although this might not have been the most effective way to push myself, it did help me prove that the money spent on my studies wasn't wasted on a mere hobby.

Another method I employed to maintain motivation throughout the years was constantly exploring ArtStation, YouTube, and Pinterest to discover what truly excited me.

Perhaps the most important thing was adapting school assignments to align with what I wanted to learn. Even if it seems like the assignment doesn't directly contribute to your goals, you can try to find ways to achieve slightly more distant objectives. Learning new stuff is what keeps it fun!

Tom van der Veeken, Environment Artist

Interview conducted by Theodore McKenzie