Recreating the Hall of Mirrors in Unreal Engine 5

Introduction

Hello, my name is Sam Hipwell, I’m a self-taught junior environment artist and I recently landed my first role in the industry at Rebellion in Oxford, UK. Here I'm going to take a look back at the portfolio piece which secured me the role and break down my process, which I documented quite thoroughly at the time. I’ll discuss lots of tips for interior environment design, using UE5’s cutting-edge Lumen and Nanite, as well as my experience mentoring with The Mentorship Coalition.

I hope this might help shed some light on how to stand out in a sea of talented junior artists. If it’s of help to you, or if you’re left with any unanswered questions, please feel free to drop a comment or reach out to me on my ArtStation.

The Hall of Mirrors Project

This is the Hall of Mirrors in the jaw-dropping Palazzo Doria Pamphilj in Rome, often overlooked by tourists in their rush between the Colosseum and the Trevi Fountain. I snapped this on my phone when I was lucky enough to explore it on a quiet afternoon in 2019. The joys of being funemployed. But let's not get ahead of ourselves.

Back in London in 2022, here I am trying to break into the video game industry, with plenty of rejections under my belt. I’m starting a new environment project and I want to push my portfolio to a level that will get studios to notice me. I’ve decided to really break down specific skills that I want to demonstrate and create a project which will condense those into one scene. I think starting the scene with intention will really pay off and avoid the trap of creating an aimless piece.

In short, the skills I want to demonstrate are:

• Using a Modular Workflow:

Breaking down a complex environment into multiple reusable components.

• Set Dressing:

Make the scene dense and rich in detail, with a focus on storytelling.

• Creating Complex Objects:

Show modelling skills, sculpt details in ZBrush and bake complex normals.

• Material Creation:

Custom tiling materials made in Substance 3D Designer, as well as the use of Trim Sheets for key assets.

I’ve been able to phase down a lot of my freelance work (I was a graphic designer at this time), which gives me three months of relative focus for the project, with the first concentrated on producing a decent first pass on my own, the second being guided by a mentor, and the third reserved for finishing off and polish. Here's the start of my plan, where I’ve thrown my ideas together on Trello, which is free:

A pitfall I want to avoid is overscoping, the art of biting off more than you can chew. A nice mantra I've heard is "rooms, not buildings", and if I’m going to consign myself to just one room, then why not go hard? I’m also keen to avoid the temptation to try to create my own concept for the project, which would risk pushing the scope too far and stall me before I even get started. So, an existing concept is needed.

Enter the Hall of Mirrors. One of the most outrageous rooms I've ever seen, dripping in Baroque swagger and flamboyant enough that showing friends pictures of it always makes them go "wow". It's perfect. Not only is it visually unique, eye-catching, and special to me personally, but it's also designed in a way that is perfect for modularity, made up of repeating shapes and objects, with plenty of opportunities to use tiling textures and trim sheets too.

Gathering References

All the best scenes begin with a reference board, and luckily the Hall of Mirrors is well photographed from every angle, so finding plenty of references is a breeze. PureRef is a free ref-gathering tool that does this really well. (Lots of industry people use PureRef to quickly slam together ideas – for me, it’s now essential to any project I start.)

From there, I now choose a key reference, which I’ve tweaked a little in Photoshop to shift the perspective and pimp out the lighting to create a concept shot to work from. (I found it helpful to focus on creating the whole scene from this one camera angle to avoid being overwhelmed with other details in the room.)

With a strong concept image in place, a healthy reference board, and a plan for the first few weeks, it’s time to start the blockout.

Blockout

My aim is to get down a basic layout of the scene's key architecture in the engine (Unreal Engine 4 for now; things get spicier later!), which is then roughly lit to provide some sense of whether the composition is working.

For some scenes, it might have made sense to fully blockout in the engine, but for fiddly architecture like this, I’ll start in Maya, for much more control. Making sure to get the units set up properly for export to Unreal, the initial blockout is as basic as possible - resisting the urge to flesh out anything. 

Modelling in Maya

By building pieces of architecture out from each other (i.e., duplicating faces to create new geometry), I can make sure that shapes snap together later. It takes some careful observation to figure out how to break down the room's shapes, but it will pay off to take my time at this stage and avoid a balls-up further down the line.

By using instancing in Maya, I’m able to project pieces along using "duplicate special" and create the full room with just a few parent pieces. Enabling vertex snapping at times also crucially makes sure that the parent pieces snap together with their instances, leaving no gaps. Maya's tools are perfect for this, and I find it seriously satisfying.

With the shapes ready, I very roughly unwrap all the pieces in Maya, which allows me to generate lightmaps in Unreal and test out some static lighting with a few bounces, which I think is well worth doing at this stage. I use the tried and tested lighting setup of directional light plus sky light and throw in the warmer interior point lights on the mirrors for good measure before a quick preview bake. Though the blockout is far from glamorous, I’m reassured that things are heading in the right direction.

Next, I'll be throwing in some basic materials and adding detail to the geometry to create the first pass.

First Pass

Now we're cooking with gas. I start adding detail to the geometry in Maya and tweaking the lighting to get it closer to my concept. At this point, any changes in Maya can be quickly exported as a new FBX and updated in Unreal, so the process of bouncing between the two feels quite natural.

Basic Materials

Building some basic placeholder materials in Unreal and also bringing textures in from Megascans and Google Images makes a huge difference, and at this stage in the project, the pace of change is really exciting. 

Pre-Made Assets in Small Doses

I want to build as many of the assets in this scene as possible from scratch. That being said, I’ll use pre-made statues, as making statues lies more in the area of character art. the Unreal Marketplace has some great pre-made ones I’ve bought, and is a great place to find game-ready assets without silly polycounts.

For now, I’m also using a pre-made chandelier, this one from Turbosquid, as a placeholder, but I will model my own later to fit the scene better. The mirror assets, furniture, and arch ornaments will provide plenty of opportunities to challenge my modelling skills further down the line, but for now, they remain quite basic.

This "first pass" stage is just about pushing the level of detail on every asset one step further from the blockout without getting too obsessed over any particular one. This idea of iteration allows me to push forward the scene as a whole, which is also a much more fun way to work.

First Custom Materials

By exporting UV maps for particular assets to Photoshop (e.g., the arches below), I can get a roughly painted texture ready. In these days of Substance 3D Painter and Designer, it feels a little old-school to be texturing in 2D, but for flat surface assets like these, it will allow me to push the quality much higher. I'm a graphic designer by trade, so this is my home turf.

From a distance, this rough paint looks perfectly fine for a first pass, and it'll be easy to add layers of detail to the painting and reimport later. 

Lighting

I’ve quickly moved away from the idea of a directional light as I want to vary the amount of light coming in from each window, as well as soften the shadows, and directional lights aren't great for either of these. For now, I'm using static spotlights instead, one for each window, with a large source radius to allow soft shadows. I'm not sure if this is the best way to do this, but hopefully, my mentor will clarify next month. For now, it looks decent and allows some nice indirect light inside, but bakes are fairly slow already. A sky light stops the shadows from becoming too dark.

Inside we've got point lights positioned where the candelabras will be, but by using larger soft-casting lights I can avoid having a light source for every bulb, which would be overkill. The bulbs are set up as small meshes with an emissive material, which doesn't contribute to the static light – nice and cheap. The bloom is tweaked in the post process volume.

Mirrors and Reflections

It's worth mentioning at this point that mirrors are (obviously) a key part of the scene. For now, I'm using box reflection captures to keep things cheap, but they're not very accurate, so I have planar reflections set up and ready to turn on. These are super expensive, but I think they'll be useful for the final captures. I've turned reflection resolution down while I'm working on things, so expect any screenshots to look janky. (Later I’ll change my mind on the planar reflections as they literally render the scene twice and when I switch the engine to UE5, I’ll have access to Lumen’s own reflection capabilities.) And so the first pass is complete!

There's plenty to do, but on a thumbnail level the scene is looking really similar to the concept already, so I'm feeling very happy with the way things are going. I'm well within my planned time frame and ready for the month with my mentor to get things pushed forward.

Trims, Materials, Modelling & Mentorship

Entering the second month of the project I have my first call with my mentor, Jobye Karmaker (Respawn, Apex Legends & Jedi: Fallen Order), who I’ve chosen from The Mentor Coalition.

In retrospect, I really couldn’t have timed the mentorship better. Starting a mentorship with a scene ready at the first pass level is a serious way to gain momentum and make the most of the experience. The Mentor Coalition wasn’t cheap, but if you’re serious about learning, I think the opportunity to have a senior artist oversee your project is easily worth it and almost unheard of in most other industries. Jobye gave me a lot of his time, really good write-ups of our calls with helpful paintovers, and I came away from each call feeling motivated and with a new direction. My main advice to anyone doing mentorships is to be prepared for every call: you’ll get twice as much out of it.

Jobye seems happy with my progress so far and has given me some great tips and ideas for pushing forward with the next stage. It’s really exciting to discuss the scene in depth with a senior environment artist, and I feel I'm in safe hands for the rest of the project. 

Switching to Unreal Engine 5

One big decision we’ve made is to switch the engine from UE4 to UE5, which should allow me more control of interior bounce lighting in realtime with Lumen, as well as let me lean heavier on detailed geometry with Nanite, which will be great for the ornate parts of the scene. 

This is an interesting one for any other aspiring junior artist. Using Lumen and Nanite means you can more or less completely skip baking light and normal maps for assets – an incalculably huge time saver and surely the future of any next-gen games. That being said, any studio that doesn’t yet use UE5 (most of them currently in 2022) will want to see that you have knowledge and experience baking lights and assets, so I’m glad I had other portfolio pieces to show this off. 

Looking back, using UE5 allowed me to push this scene forward infinitely faster and to a much higher visual standard than I would have been able to with UE4. No regrets whatsoever, and I hope I get to work with UE5 professionally soon.

Fleshing Out Materials

By creating the brick floor texture from scratch in Substance 3D Designer I’m able to take control over every aspect of its look, adding detail to the bricks like air bubbles and cracks, as well as fine-tuning the colour and the brick separation. There's plenty of great tutorial content for bricks in Designer, so it’s just a case of taking control of the process to suit my own reference, starting with the herringbone pattern.

I've designed the ceiling to be a nice flat unwrap so it can be fully textured in Photoshop, where I'm confident I'll be able to build a nice painting and merge in lots of Baroquey figures and details. I’ve thrown this together to get things started and then I’ll slowly incorporate more details and images from reference.

It's interesting how much difference updating these two materials makes to the bounce lighting and immediately brings the scene closer to the reference, with a few tweaks to exposure and balance of the lamps' intensities.

Trim Sheets

Chatting with Jobye helped me realise I can break down more assets into trim sheets than I'd thought. I'll be reorganising textures to make the most of this. I'm building my first sheet, which I'll use to add trim to the brick floor. After creating a new straight variant of my current brick texture in Substance 3D Designer, I add some geometry to the floor piece in Maya and carefully unwrap it to make use of the new texture.

On this sheet, I will include textures for the wall arches and window panels. The layout can be added to as I work.

I also create a combined texture sheet for a few awkwardly shaped panels to make the most of the space.

Modelling Ornate Detail

Time to tackle that ornate detail and flesh out all of my assets for the second pass. I'll be using live surface quad draw in Maya to lay down the base shapes, paying attention to planar changes, then extruding and leaning heavily on the crease tool to get the shapes to pop without introducing silly amounts of geo. Jobye pointed me to this tutorial for ornate trims. (I think this excellent tutorial helped me so much to model ornate shapes in a sensible way, and I gained a huge amount of confidence from it. Well done Jobye.)

Blends, Sculpting & Adding Detail with Nanite

Vertex Blending

To help the placed assets really feel grounded in their environment, I'm making a new, dirty floor texture variant which I'll blend using vertex painting in Unreal. This will make a subtle but really important difference and really help reduce that "gamey" feeling. (In retrospect, I could have used decals for the grime to save time here, but it was a good skill to demonstrate, and the end result looked nicely blended).

Adding Detail with Nanite

A large part of this stage involves heading back into Maya and adding another level of detail to each asset. I'll use plenty of the Crease tool as well as lean on things like EP Curves to generate nice braided ropes and cables. With UE5 I'm now importing smoothed meshes at a high level of detail, without ever baking normal maps: Nanite is insane! It's speeding the process up so much, not to mention Lumen meaning I don't have to rebake lighting, like ever.

Jobye tells me 500,000 polys per asset is a sweet spot for detail and import times for Nanite, so this is what I'll be aiming for when it comes to high-detail sculpted assets. For medium-detail smoothed meshes that lie somewhere in the middle, I’ll be well below that level, but UE5’s documentation suggests it’s still worth enabling Nanite anyway and save me from wanting to bake them down any lower.

I was pretty daunted by this table, but after I broke it down into its key shapes and used quad draw to lay them out, then extruded and used soft selection to add depth and curve, this turned out pretty nicely for a base mesh.

Sculpting

This next stage is pretty ZBrush-heavy, adding fine detail to the ornate assets and really giving them that extra 10%. I’ll subdivide a lot to give me plenty of polys to sculpt, make good use of masking in ZBrush for harder edges, try not to lose track of reference, and won’t overcomplicate the brushes – the basics will do. My mentor Jobye was really helpful at giving feedback on my sculpts here.

I’ll use Decimation Master to reduce all my sculpted assets down to my 500k sweet spot before importing with Nanite enabled on the asset in UE5’s content browser.

Material Layer Blending in Unreal

To add some pop to my gold material I'll need to bake AO and Curvature maps of these high poly assets. After a bit of tedious unwrapping of the decimated ZBrush models in Maya (Select Shortest Edge Path Tool is useful for this), I bake the maps in Substance 3D Painter, using "high poly as low poly", and then use these textures to blend between different Material Layers on the model in my Unreal shader. The textures are treated as parameters so each object can be material instanced and new maps applied and tweaked.

The result is that the ornate gold assets no longer have a flat and lifeless material but have much more character and feel a lot more grounded. I'll be working on this material a lot before I apply this approach to all my other assets.

A Little Problem Solving

An interesting problem that some handy people on Reddit have helped me solve is that I have "faked" the candelabra lighting, and the orbs show up in my mirrors' reflections. The solution to this problem was to change the object's sensitivity to particular light channels; by disabling lighting channel 0, the orb no longer shows up in the reflection but does cast light on the geometry. Nanite isn't yet compatible with lighting channels, so I have to separate the mirror's surface from the ornate frame before doing this.

Implementing Feedback, Lighting Passes with Lumen, Story

Feedback Time

With my mentorship finishing and the scene approaching the first completion stage, it’s a really important time to get lots of feedback, even though it’s nerve-wracking to share the piece before it’s finished. I’m posting on Discord pages, like ExperiencePoints, The DiNusty Empire, and The Mentorship Coalition’s own private Discord page, as well as reaching out to some of my favourite artists on ArtStation. 

It’s honestly amazing how many artists take the time to respond when you approach them sincerely for criticism on a WIP. In the spirit of this, please feel free to contact me on my ArtStation if you ever want my humble opinion on a portfolio piece; I’ll be happy to reply as soon as I have a minute.

I think that receiving and acting on feedback is an essential skill for a junior, and it’s something that employers really like to see in interviews, so be bold to ask and be grateful to be given it, no matter how much you might disagree with someone’s opinion.

After sharing some screenshots from the scene, a few smaller issues have come up, like materials needing work or trims needing adjusting. No problem. But several larger issues were brought to light (the pun is clarified soon), which would help really drive the scene forward towards completion.

Lighting with Lumen

The scene really needs a full re-light as I’ve been building the space with a need to see all assets well-lit, so the lighting was too flat or blown out or just generally holding back the realism. Below shows my progress in figuring out the new lighting situation, heading (I hope) toward better subtlety and realism while also introducing more dark areas for the eye to rest in such a busy room. (This was something that my eyes had gone a bit blind to, a really good example of where feedback is essential.)

Using Lumen means all lights are now movable rather than baked, and the room gets filled with bounce light without ever having to bake. It’s a really natural and easy way to light and allows me to quickly build a strong setup without any guesswork. The only downside is that it’s pretty expensive, significantly slowing performance as the lights increase in number. As this is a portfolio piece and it’s still running well over 30fps, I won’t be worrying too much about this cost, but converting the lighting to a traditional bake would be a clear performance choice if this were to be part of a game.

Storytelling

At this stage I find myself asking: is my environment really telling a story or is it just an empty (albeit pretty) room? Although a lot of my feedback thought that the room told an interesting enough story on its own, there was one particularly useful comment urging me to ask "what happened here?". (As I realised at the time, this was by far my most important feedback and something I’ve promised myself I’ll never forget.)

To answer this question, I need to double down on the story being told by leaving clues and points of interest for the player. With a bit of a search, I soon find out that the Hall of Mirrors in Palazzo Doria Pamphilj has been used for events on more than one occasion, one of which was a fashion show.

This immediately caught my attention and it doesn’t take long to identify which elements I need to add to tell the story of a catwalk/runway event having happened here the night before. 

By creating some fairly simple new assets, like the spotlight rigs, the runway carpet, benches, discarded bags, cards, etc., I’m able to finally answer that fundamental question – "what happened here?" – and add a lot more interest to my scene.

An important part of this shift in storytelling means the lighting scenario needs further work to balance the interior lights, make increasing use of volumetric lighting to add that smoky mood, and make the difficult choice to tone down the lighting on some of my most precious assets. This will all benefit the scene and the story, which is by far the most important thing.

It's a balancing act of leaning on Lumen's indirect lighting while tweaking the intensities and attenuation radii of my various lights to create some nice values which illuminate my key areas of focus. The temperature of the lights, as well as their volumetric contributions, are also very important to the scene's mood.

Final Level of Detail

At this stage, it's a case of adding some polish and crunching out the last of my assets by detailing them in Maya and baking blend maps in Substance 3D Painter before hooking them up for material blend instances in UE5.

I'm also making pretty generous use of decals to add little flourishes, footsteps in the carpet, grime, and grunge to the scene, which is much more efficient than creating entirely new material blends where they aren't needed.

Once this is finished, I'm pretty much done. I've just about spent the 3 months that I planned for the scene, meaning with a week or two of polish I'll only be slightly behind the schedule that I planned for myself back in April. I'm really pleased that I didn't grossly overshoot and I've achieved everything I set out in the plan, probably to a higher standard than I'd initially hoped. 

I've loved building this scene; I'm really proud of the result and I've felt like the process has pushed my learning in interior environment art in all sorts of really exciting ways.

Conclusion

Thanks for reading! To any of the many who helped me with feedback, you are beautiful people, and I’ll always be grateful that you took the time to help me out. Biggest thanks of course to Jobye-Kyle Karmaker, it was a pleasure. Thanks also to Kirill at 80 Level, it is an incredible resource to which I am very lucky to be contributing.

Cheers! Maybe I’ll see you somewhere down the road. Drop me a line on ArtStation or LinkedIn.

Sam Hipwell, Environment Artist