The Shape of Water: Lighting in Toolbag and Texturing Workflow

Introduction

Hello, my name is Jeff Martin and I am a 3D Artist at Ubisoft Toronto. You can find my portfolio by following this link.

My career began in mechanical engineering at Queen's University. After working a few years in the field, I realized a career change was needed. I went back to college and studied video game development and design at the Toronto Academy of Design. Upon graduating, I spent a few years working on my skills and portfolio before applying to Gameloft.  Then, I worked at Gameloft Toronto for 3 years in the mobile industry and successfully shipped three projects: “CIS Slots”, “The Blacklist: Conspiracy”, and “Disney Magic Kingdoms”. My roles in these projects were to create 3D environments as well as refine the lighting pipeline for offline rendering.

I thoroughly enjoyed my time at Gameloft, however, my true sights were set on AAA gaming.  After further 3D practice catering to AAA art, specifically the PBR pipeline, I applied to Ubisoft Toronto and have been working there for the past 3 years. I worked on “Watch Dogs Legion”, for 2 years.  Currently, I am working on an unannounced title. My time at Ubisoft has allowed me to grow drastically as an artist and learn industry techniques I was not exposed to at school.

Why Environment Art?

Environment art has always been my passion and I have worked as an environment artist for approximately 6 years.  I enjoy realizing and creating environments that are believable as well as lighting and set dressing them. I tend to get lost in them and forget about my surroundings, as weird as that may sound.  My main inspiration was my experience playing AAA games and wondering how the art was created. Seeing all the great art on Artstation made me jealous and increased my drive to produce art of that quality level. 

The Shape of Water Project: Inspiration

The “Shape of Water” scene began with a small discussion with a friend about working on a side project. We didn’t know what we wanted to work on, so we began building mood boards of various concept art environments found online. It then evolved into us deciding on replicating a small scene from a movie. I personally like moody, grimy scenes based on photographs as opposed to concepts. This narrowed our searches down and we came across the “Shape of Water” apartment photos. We had both seen the movie and enjoyed the set dressing and lighting design. The decision was easy at that point and we began to collect as many references as possible for that section of the apartment. In addition, we watched the opening scene of the movie over and over, which showcases the apartment. It took about 3 to 4 days of studying the scene before we committed to starting production. Our main goal was to replicate the scene as close as possible and capture the mood of it. The lighting and material treatment were key to accomplishing this. 

Modeling Workflow

The production of the room began with a block out / grey box of the room with a basic lighting pass. I camera matched the scene in Marmoset to the photograph of choice. It was important to capture the proper scale of the scene before doing any props or texturing. Once the scene was locked down, I textured to completion the walls, floor, and window. I believe this is the foundation to selling the scene. If the structure does not look good, then no number of props will save the scene. I oversaw all the assets, however, my colleague provided modeling support along the way.  

To summarise the workflow, it would begin with modeling each asset to scale based on the reference and the relative sizes of objects. No texturing was started on the assets until all sizes, proportions, and likeness were locked down  Almost all the art was created in 3ds Max. ZBrush was used for organic sculpting of damage on the chairs, wood edgewear, and some ornate details on the hanging light, lamp, and table sides. All texturing followed the PBR workflow and involved a combination of Substance Source materials and custom painted maps in Substance Painter. The walls and floor were textured all in Photoshop by blending images and layering grunge to build up damage at the desired locations. It is important for large surfaces such as walls and floors to not look repeated even though they may use tillable textures. The top priority for these surfaces was to make them look unique. Decal alpha floaters of damage and dirt were hand placed in Marmoset to break up any texture repetition.

I don’t want to get into too many details on the modeling side, but the focus was on details, not budget. The detail lighting render (see below) shows off the general modeling shapes. All edges were chamfered to reduce the “3D” look. This would allow light to catch all the surfaces and create believable highlights. Pay close attention to chamfer sizes to try and match real-world references. Too large of chamfers can make an object look “cartoony” and too small of a chamfer can cause bad aliasing. Generally, for game assets, you want to exaggerate the chamfers a bit to allow light to catch the edges better. Any curved surfaces were made sure not to look faceted. It's not necessary to turbosmooth everything but rather add enough edge loops to have perfect curves based on the size of the object. 

If an asset had multiple elements, for example, the lamp, the model would be broken up into the base, bulbs, and lamp shade and exported separately. This would allow different materials to be applied in Marmoset and tweaked independently. All the props were uniquely unwrapped in 3ds Max with no tilling, allowing for custom painted edgewear, dirt, etc. Any small details were normal mapped on.

Glass Material

I'd like to take the time to focus on the window treatment. Glass is a tough material to get right. Luckily, Marmoset makes this process relatively easy providing you give it the proper maps. I began with the default glass shader and added the transparency profile (see below). 

It is very important to have a good glass reference to try and mimic. In this case, my main objective was to allow light to pass through, see the outside details, and allow for internal reflections of the emissive lights and props. A greyscale map was created to tell Marmoset where to let light through and plugged into the Mask channel of the transparency tab. Darker shades of grey would indicate less light transmission and simulate the dirt dripping around the window edges. The Specular map controlled the strength of the reflection of interior objects on the glass surface. In this case, the lighter the greyscale, the higher the reflectivity. Darker greys were reserved for the dirt. A few grudge maps were layered to give some variation to the reflectivity across the glass surface.  

Once the specular and transparency maps were refined, the Gloss map was created, which drove how “sharp” the reflections were and acted to blur anything behind the glass.  Lighter shades of grey blurred anything behind the glass while darker shades tightened up the reflection of inside objects. The final map, Albedo, was used to color tint the glass to better match the color temperature of the moonlight and further darken any dirt areas.  The great thing about Marmoset shaders is that once you plug in your maps, you can still refine them with the sliders in each tab. I found a glossiness of 0.55 provided just the right amount of background blur and reflection sharpness. I set the IOR to be the default of 1.0 as higher values distorted the refractions too much. Make sure to have micro surface checked on under the transparency tab to allow for blurring. As well, have Albedo Tint checked on to allow the albedo map and color to drive the glass color. A normal map, in this case, is optional and adds a subtle distortion to the glass. I experimented with raindrop normal maps try and make it look like water was on the window but ended up settling on subtle water streaks. 

Texturing

Texturing is the most enjoyable part of environment art in my opinion. I have embraced Substance Painter and use it exclusively to create the custom painted textures. Photoshop is only used after for level and opacity tweaks. The walls and floor were the only elements textured in Photoshop. Most of the materials for the walls were obtained from Textures.com, a great resource for PBR textures. All props were custom painted in Substance Painter using pre-set generators such as edge detection, gradient buildup, and dirt buildup. To get the best results, bake the following maps as seen in the screenshot. 

Ambient occlusion will drive the dirty buildup in crevices. Curvature will drive edge damage and wear. Position will allow for directional dirt. Thickness can help control dirt buildup for some of the smart materials. Experiment with different smart masks and materials to give the best result. Unfortunately, there is no strict rule to follow and does require a lot of tweaking.

Albedo and Roughness maps were generated from Substance Painter (see below).

I feel a good, artistically strong albedo map can really help sell an asset. Color variation across wood or metal surfaces for instances instead of solid colors really helps. Grey scale variation in roughness maps is equally important. Darker areas will result in sharper reflections while lighter shades will result in more glossy reflections. It was very important for me to match the material types in the movie references. The difficulty is trying to figure out the proper albedo values and colors without the influence of lighting. The reference scene is yellow tinted and teal which can trick artists into thinking all albedo maps need those tints. The lighting in Marmoset will drive the color of the final look, so try and stay true to the colors of the original unlit materials. Some of the generators I used can be seen in the following screenshot. Most notably, I used the Ground_Dirt for directional dirt buildup and Edges_Uber for edgewear.  rime_Cavity is also very effective at creating dirt buildup in crevices. Some key filters used were histogram shift and warp.

The posters were just images found online from the old 1950-1960s era movies. I chose the “Creature of the Black Lagoon” movie to loosely pay homage to the “Shape of Water” plot. A roughness map and opacity cutout were added to break up the surface and give the effect of torn away areas. The following screenshots show the transparency settings needed to create a torn paper effect and the associated map.

Preparing the Render

The final render process was exclusively done in Marmoset Toolbag. I chose this toolkit because it provides real-time realistic feedback for tweaking materials and lighting. I feel the visual quality it produces is on par with Unreal 4 and is simple to learn as a standalone program. Omni lights were used for the lamp and ambient lighting. These create nice dispersed light in all directions with soft shadows. Spotlights were used for the hanging pendant light and back left wall light (off camera). These create more direct targeted light. A directional light was used for the moon and outside light effects.

To get the right atmosphere was mostly trial and error with the lighting. Intensity values were tweaked to get the proper amount of illumination while maintaining contrast in darker areas. In Marmoset, some important sliders to experiment with are distance, attenuation curve, and shape width. Distance controls the area of influence of the light. Attenuation curve affects the fall of the light. Shape width gives the light volume and causes more natural blurred shadow falloffs.

The following shows the render settings I used to get the best results. These settings will differ depending on the scale of the scene but are a good baseline to follow. Make sure to increase the sampling quality in capture settings to at least 100x to reduce aliasing and noise.

The following settings are what I chose for the camera setup. Tone mapping was used to color grade the image and give it the warm tint. It also helped with tweaking the exposure limits to brighten underexposed areas of the scene. 

To summarize, the most important aspects of a believable render is the lighting and colors.  Light intensities should look natural and not overblown. Shadows should not look too sharp and have natural falloffs. Color temperate should match real-world lights. Often, trial and error is the best approach when lighting a scene. Stay true to the mood of the references and trust your eye to gauge what looks natural. Throughout the lighting and rendering process, I would often render out an image and bring it into Photoshop to compare it side by side to the target movie reference. This back and forth process gave me great results. 

Afterword

I hope this dive into The Shape of Water environment has helped many of you to learn more about techniques to improve your personal projects and motivate you to keep practicing. I didn’t want to go into too many details and over complicate things. I chose to focus on the key elements that sold the scene, namely the lighting, materials, and glass. I am always looking to improve with every new project I complete. Feel free to contact me on LinkedIn as I am willing to help with any questions you may have as well as network with fellow industry professionals.

Jeff Martin, 3D Artist at Ubisoft Toronto

Interview conducted by Arti Sergeev