From Film to Unreal: How to Make 3D Scenes Look Like Black-and-White Photos

Introduction

Hi everyone! I'm Scott McGrath, and I'm a Lighting Artist in the games industry, currently working at Halon Entertainment. After I graduated from university, I worked in the live-action film industry for ten years, focusing on cinematography and lighting. Eventually, I was ready for a career shift, and I returned my gaze to my love of games. Once I learned about Lighting Artists, that became my new goal, and I've been locked into that path ever since. I took a Lighting for Games class at CGMA, built out an initial portfolio, and was fortunate enough to get hired at my current position a few months later. It's been an amazing journey, and each step of the way has been fueled by a passion for those intersections where art and science meet. 

I'm always hungry to keep learning and expand my skills and knowledge. To that end, I enjoy continuing to work on personal lighting projects, as time and energy allow. Recently, I completed the Simplistic 4, a lighting prompt given to me by a wonderful mentor, Tomasz Bentkowski. He provided reference for four simple images to interpret and replicate. Let's dive into that one!

Translating Traditional Lighting Techniques Into UE5

My experience in shooting and lighting live-action projects really gave me a lot of initial momentum when it came to learning lighting in Unreal Engine. In film, I was almost always working on location, not in studios – "faces in spaces" was the game, and I found the general approach to lighting in real-time and 3D to be quite similar.

Working on film sets, especially on location, you are always working against the constraints of time and budget. Sometimes, the more your team fights that flow, the worse the experience and result can become. Instead, you learn how to keep things simple and keep your footprint to a minimum, as much as possible. When we're employing a high-level approach of focusing on what's essential to a scene, that way of working can come more naturally and help things flow both narratively and technically.

On an abstract level, it's really quite a relatable philosophy to film: always be listening to what the story, characters, and environment are telling you. What's important in this scene or moment? How do we separate certain characters or set pieces from this greater world with lighting? What's the tone and emotional state of our POV? What are the practical light sources in this world to motivate our lighting?

When it comes to the nuts and bolts of accomplishing the look, it's not surprising that a lot of techniques are used similarly between mediums. And a lot of the tools are very familiar! Spot lights, rect lights, point lights, global lights, bounce cards, practical sources – all of these translate very nicely from the real world, as intended. 

In regard to workflow, establishing the key light's qualities is the primary step – its direction, intensity, motivation, source size, color temperature, etc. I believe this is a step that beginners should always spend longer on than they think is necessary. But it is always the root and foundation of a scene's lighting, and getting that dialed in helps make the whole rest of the process flow much more smoothly and naturally. From there, it's a matter of finding the right amount of separation and depth with rim lighting, using fill and bounce to hit your ideal contrast ratios, and eliminating hot spots that distract from the focus of your frame. I think that the WYSIWYG nature of Unreal Engine 5 makes it easier than ever for beginners to jump in, start playing with lighting, and immediately see the results of the decisions they're making. 

Working With References

For this project, Tomasz selected four reference images for me to replicate the look of. The goal was to focus on lighting some "simple" scenes and continue to develop my understanding of how to best translate my live-action experience into working with 3D environments and materials. They were chosen for their ability to be quickly built out using just primitive shapes in the engine. And black-and-white was chosen to help the focus stay on luminance, contrast, etc. I coined the project Simplistic 4 almost a little ironically, because they were all their own little rabbit hole to dive into. But that's what made it a fun challenge!

Four reference scenes:

Very early into lighting, I realized that I had a choice to make: is the challenge of this exercise to emulate just the essence of these references, and see how close I can get with just a simple and "realistic" lighting setup? Or is the goal to try to match the reference very closely and accept the possibility of over-complexity? I opted for the latter, but it was still (and always is, I suppose) a balancing act.

When looking at any reference images, it's important to keep in mind that they (most likely) ALSO faked it. It's easy to initially assume that these references are just photographs, straight out of the camera, lit only by natural light. Ok, with the exception of the more abstract stairway image. But we cheat in film, we cheat in games, heck, we cheat on our Zoom calls. With reference photographs and stills from films, this cheating could take the form of artificial lights being used on set, or it could be touch-up and correction done in post-production. Everything is just a reference of a reference of a reference. That doesn't mean that it's not important to understand the fundamentals of physical lighting (it's critical!), but if your intent is to emulate a particular image, it's good to keep that in mind.

Let's take a look at the reference image for "PlantHallway" as an example of things to analyze. For starters, you can see that there is a very strong light source coming from two different directions. There could be a number of explanations for that, and we can only conjecture on what we can't see outside of the frame.

The Lighting Setup

The foundation of each scene's lighting is a pretty basic setup using vanilla UE5 actors. I utilized Dynamic Global Illumination and Reflections through Lumen. To establish my main key light and ambience, I used a Directional Light, Sky Atmosphere, and a Skylight. The Directional Light intensity was set in the ballpark of real physical values, and the Skylight was set to capture the scene in real-time, no HDRIs. Even though I wasn't intending for this to be a strictly physically-based exercise, using a PBL starting point got everything "working as intended" out of the box, and allowed me to manually set my exposure and establish a foundation for the look very quickly.

To further the lighting from there, I utilized a variety of local lights, light blockers, and the occasional bounce card. Most of the local lights were very broad rect lights to provide more fill or help smooth shadow falloff in certain areas. Shadows are ray-traced, and since these are just static compositions, I increased the 'Samples Per Pixel' on most of my lights to help minimize noise and artifacts.

Here is the "PlantHallway" shot before and after local lights were added:

For the "LonelyBall" scene, the reference shows a very broad and soft light coming from the screen-right opening. To drive this look, I rotated the Directional Light so that it is not shining any direct light into the scene. The Skylight then throws indirect light into the room, and I then exposed my camera to that level of illuminance. However, the Skylight alone was a bit too soft – it lacked a feeling of direction, specularity, and the contact shadows from the ball lacked any crispness. To augment the Skylight, I placed a bounce card outside of the frame. The Directional Light is slamming into that bounce card and providing a slightly more punchy bit of indirect light.

Since basically all of the light hitting the scene was indirect, it was still lacking some specularity and directionality, especially on the ball. I added a Spotlight to help bring just a bit of that into the mix.

Before and after:

Finally, I added some light blockers to help add some contrast and shape the falloff on certain parts of the scene. The light blockers are just primitive planes with a black matte material applied. Since they would have appeared in frame, I enabled Hidden in Game, Hidden Shadow, and Affect Indirect Lighting While Hidden on the planes.

Before and after:

Post-Process

In film, it's a cliche that many student and amateur filmmakers choose to make a black-and-white film because they think it will automatically make it look cooler. Unfortunately, this often has the opposite effect, because a good black-and-white image relies on so many of the things that novices have not yet mastered. Rendering monochromatic compositions really does lay your lighting bare. Without color information, you're exposed to the shortcomings of your own craft in terms of portraying realism through layering and depth.

In this project, I was pretty stubborn about trying to accomplish this primarily through lighting and materials rather than through tone mapping curves and heavy post-processing. I didn't touch Unreal's default tone mapping, but I did end up doing some minor color-correction, just to take it that last 5% or so. While I felt like I was prepared to tackle these obstacles through lighting itself, it very quickly became clear just how dependent I would be on shader manipulation to get me the rest of the way there.

The Role of Materials and Surface Roughness

Ho boy, let's get into materials! This was, by far, the biggest hurdle for me during this project. And it was probably the part that I spent the most time on because… I am not an environment artist. In my previous personal projects, I could generally do enough to do some set dressing, tweak some shaders, etc., to get it to "good enough." But that was about where my skills ended. After sending one of the rounds of my renders to Tomasz for review, he informed me that the lighting was just about there, but that I needed to add a lot more detail and variety to the surfaces. The goal was to increase the sense of natural realism but adding more variety to Roughness and Albedo values, as well as minimizing any noticeable texture tiling.

In cinematography and lighting, you are at the mercy of what's in front of the lens: set, costumes, props, etc. So even though this was ostensibly a 'lighting' challenge, it was time to up my environment game. These scenes were all initially constructed using primitive cubes with Megascans textures applied. And even though Megascans are relatively detailed, they can appear quite flat and unrealistic when simply applied on their own to flat cubes.

I picked some brains of people much more skilled than me, and I came away with the suggestions of implementing some detail textures, displacement, and decals. On most surfaces, I added a layer of both Albedo and Normal detail textures, and that was already a big improvement. Then, I incorporated some Nanite displacement and material blending via vertex painting. Easy Mapper, a UE5 tool created by William Faucher, was a simple and convenient way to get started with that. His master materials allowed me to easily incorporate displacement, blending between three different materials, a puddle layer, etc. Vertex painting allowed me to be really specific with the art direction, and I could control the placement and intensity of the blends with precision. Finally, adding a few decals added that last bit of variety and storytelling to help make the spaces feel alive and 'lived in.'

Here are a few renders, with more-or-less final lighting, before and after applying these techniques. You can see how game-y they felt originally! Before:

After:

All together now, here's a little WIP timelapse of all four scenes, with both lighting and art iterations:

The Camera Setup

To capture these scenes, I used Unreal's Cine Camera actor with a very basic setup. I brought the camera into the level, locked it in place, and set the crop ratio according to the aspect ratio of the reference image for that scene. Once I had mostly blocked out the scene with primitives, I would check the reference image to get a feel for the focal length they used to capture it. I wasn't very scientific about this step; I simply looked at the reference, made a guess based on my experience, and then adjusted my focal length from there to get the level of depth compression to more or less match.

I did not bother to use physical camera exposure settings for this project. Once I was happy with my first step of lighting, establishing a key light, I set my camera's exposure mode to Manual and adjusted the exposure compensation to what felt right for the key light and base exposure.

3D & IRL Cinematography: The Differences

Earlier, I mentioned that, much like in film, simplicity is often key, both to the process and the result. When it comes to visual storytelling, we sometimes ask, "What's the more interesting choice?" Oftentimes, it is not the most complex choice. But in live-action, there are frequent occasions where you wish you could just cheat the laws of physics or evade the constraints of time or manpower, and do a little more, sneak another light in here, block the sun more fully there, etc. 

But in 3D, you have almost unlimited power and control (and time, if you're working on personal projects). This is a blessing and a curse. Want to throw some fill light up into the ceiling? Easy. Decrease the sun's intensity in certain parts of your environment? Done. Attach a moving light to an actor in your scene? No problem. I found myself constantly iterating by throwing more and more into the scene, drunk on the power of being able to illuminate every pixel. But many times, more is not better, and just because you can, doesn't mean you should. This has been an interesting lesson to re-learn during my career transition.

If you're coming from film, another major part of lighting that takes adjustment is the use of bounce lighting and diffusion. With film and photography lights, we are almost always placing equipment in front of light sources to soften them (diffusion). There are a staggering number of different flavors of diffusion gels and fabrics, all of which can dramatically change the specularity and shadow quality of the source. Want your small light source to feel like a large source, with lots of wrap and soft shadows? Put up a 12'x12' frame of full grid cloth. In doing so, you're effectively changing the source from the lamp to the diffusion frame, so it behaves like a much larger source. In Unreal, there isn't really a way to emulate diffusion like this out of the box. You can, however, use "nets," which are light blockers of various opacities, with some very minor shader work. If you expect translucent materials to behave like diffusion, you will be just as disappointed as I was! Instead, the size of the source is adjusted directly through the light's properties (width and height for rect lights, source radius for directional, point lights, and spot lights), which takes some getting used to. Not to mention you can adjust both the specular and diffuse multiplier (and break PBR, in the process!) of the light, if you so choose.

Similar but different, bounce lighting is a little bit more comfortable of a transition. Lumen has brought dynamic global illumination to the masses, and with that, bounce lighting more or less works as expected, albeit with caveats. Lumen (let's leave path tracing out of this discussion) is much more of a sledgehammer than a chisel. It's great at handling the broad strokes, but doesn't have the nuances of physical indirect lighting. Much like diffusion, folks lighting for film and photography employ a bazillion different tools to bounce light with different qualities. From sharp and shiny mirrors to perfectly matte and rough surfaces, all of these bounce card surfaces produce different results. In Lumen, the indirect light produced from the bounce is great at contributing to diffuse values, but won't really bring out specularity, regardless of the material on the mesh used to bounce the light.

Scott McGrath, Lighting Artist

Interview conducted by Arti Sergeev