Crafting a Cel-Shaded Game-Ready Scene in UE4

Kate Blackshear showed her first environment in the style of Little Nightmares with a cel-shaded look. The project was made with UE4 and Marmoset Toolbag.

Kate Blackshear showed her first environment in the style of Little Nightmares with a cel-shaded look. The project was made with UE4 and Marmoset Toolbag.

Intro

My name is Kate Blackshear and I’m about to graduate from SMU Guildhall’s master’s program. I came from a customer service background, made a drastic career swap from a well-paying job and a secure lifestyle to starting over in graduate school for 3D modeling. Game development was something I’ve always been interested in, but I wasn’t sure if I’d fit or where to even start – I just knew I needed a change, so I made the plunge and have not regretted a moment of it.

I’ve worked on three titles so far, one is successfully published on the Steam store, named La Rana. Right now, I’m in the midst of talking to several studios and hope to move on to a place where I fit right in!

Creeping

Project Idea

The game I created the scene for was originally called “Night Terrors”, closely based on Little Nightmares, but with more whimsy and light-hearted combat tactics. I had created a rough sketch of how I envisioned the environment for this game and held onto it as a charming idea even after the pitch was canceled. The player was a small creature that had to traverse the area with a blanket and a flashlight from one side of the room to another. The player would be about the size of a small mouse in a big room, which made for an interesting environmental design. I had the choice of doing a life-size room from the perspective of the player or an isometric view, and I thought it would be much more interesting to have an isometric view so I could play with the edges and bounding box if you will. The important elements of the production were keeping the initial idea but knowing I was free to venture in a more refreshed direction. I do not think that this fits the game as it was intended before, but I also do not think that is a bad thing.

The important elements of this production were trying to instill a similar mood to how I envisioned the original pitch and to create an aesthetic that looked illustrated instead of realistically 3D modeled. Focusing on these goals let me experiment a great deal without losing sight of what I wanted the final piece to look like.

Production: First Steps

I started production by blocking out the room with very simplified shapes. Funny enough, they didn’t change too much from the beginning to the end, because I knew I wanted the meshes and geometry to be simple in form. I also kept in mind that I needed a way to tie everything together outside of my texture work, and I decided to do that with chamfering. My goal was to make everything as simple looking as possible so that the messy or painterly texturing would not clash with the geometry.

The most challenging thing I faced, but what was actually for the better, was changing the bounding room. Initially, I had a basement room that had a low ceiling, a small window, and felt very dreary, but it was suggested to me by a professor, Boris Fisher, that I could make the room into something more, like an attic. Trying to figure out how to pivot and how to recover for lost time was a big challenge, but I found enough reference of what I wanted the room to look like that it helped make up for lost time with concrete ideas.

Texturing

This is my first overall stylized environment, and I was very much unsure how I wanted to approach the texturing. My comfort zone lies in a dirty combination of realistic textures and painterly details, so I decided to test this out in the engine. When I did this, I immediately saw that they were not working with the way that I modeled 3D objects because they looked too serious and realistic for such simplified, light-hearted models.

Unsure of what to do, I decided to move forward with bakes, blocking out colors and playing with lighting and post-processing. I wanted the colors to be a bit on the “odd” side, so I spent the majority of my time trying to find colors that complimented each other but still felt a bit off and weird, similar to the color schemes I use in my drawings. My post-processing came in handy a lot because I was able to adjust the saturation of the shadows or hue of the overall scene.

There was a point I found a color scheme I was really happy with but I decided to keep experimenting, essentially overworking the piece, and thought I ruined it because I changed it too much. I actually thought there would be no coming back from this, but I ended up adjusting the textures and lighting enough to make something different and arguably better. It had been the first time I felt I overworked a 3D piece, so it was really disheartening at that moment, but knew I could bring it back to something appealing if I did it once before.

I come from a strong abstract painter background and I think that shows a lot in this piece. I tend to go for scratches or painterly splotches of color in a lot of my work. It’s almost a trademark at this point if you see little dots splattered everywhere. It creates a lot of visual interest without morphing the area into something else or changing the mood. Although it’s suggested to not use a lot of variation with roughness maps when working in cel-shading, I decided to use this style I have and put splotches in the roughness map to interact with the lighting and create some visual interest outside of the albedo maps.

I’m not well-versed in technical art, but I used a tutorial on YouTube in order to get the cel-shading effect. It was a post-processing shader combined with an outline shader and a sharpen shader.

Lighting

1 of 2

Lighting is arguably what I believe to be the most important part of a scene. Approaching this scene with cel-shading, I knew that I wanted to accentuate the look with hard lines in the edges of the lighting because the most memorable aspect of a cel-shaded scene for me is the lighting. I wanted to have a lot of color contrast in the scene as well, so I kept as few lights as possible, with most of the light coming from the lightbulb in the center of the room. The biggest struggle I had was wanting certain things to be lit, but knowing overall the piece looked better with a certain amount of lighting hitting certain places. Additionally, I wanted the shaders to play well together, so that created a bit of a challenge that I eventually overcame with iteration after iteration.

My biggest piece of advice I can give for lighting is get everything in the scene as soon as possible, get the colors blocked out and start playing with lighting and post-processing. I would even go as far to suggest to get a few lights in, play with what you want to change in post-processing, and then go from there with the regular lighting and colors because you can get really interesting looks from keeping an interesting post-process and merely changing the lights. And of course, I hope it goes without saying, bake lighting as much as possible! Lighting should take just as long as modeling out everything.

1 of 2
1 of 2

Baking

I did use Marmoset Toolbag for the baking in this project. Initially, I did use it as another viewer because I prefer the way it renders over Unreal Engine 4, but once I decided I wanted to play with different shaders and post-processing, showing it in Toolbag was off the table and used only for baking; testing out how anything looked in Toolbag would have ultimately been fruitless in comparison to UE4. I try to be very simple with my pipeline for Toolbag by importing each prop at a time and baking, splitting everything up into their own output folders for organization purposes. I only bring in all the props at the same time if I’m looking for more depth with Ambient Occlusion. I bake Ambient Occlusion and Normal maps within Toolbag, and occasionally I use curvature maps to bring out hard lines of edges. I combine the AO and curvature maps in the albedo in Photoshop so that there is more depth and detail in the final render.

Сhallenges

Funny enough, my biggest challenge was the overall texturing. It was the first time I had worked with a 2048×1024 texture map for all assets in the scene, and I had not planned it out well. In my head, I envisioned asset after asset and had trouble knowing when enough was enough before everything was textured. This was one of my first stylized scenes, so I was anxious to texture, but I wanted to tell a story, and I thought if I add more props, then more of a story will come out; the story didn’t come until the finishing touches of the writing on the wall and lighting and overall style was tied together.

1 of 2
1 of 2

Kate Blackshear, 3D Artist

Interview conducted by Kirill Tokarev

Join discussion

Comments 0

    You might also like

    We need your consent

    We use cookies on this website to make your browsing experience better. By using the site you agree to our use of cookies.Learn more