Turning a 2D Illustration into a 3D Environment with a Hand-Painted Look

Introduction

Hi, my name is Shuming Zhang, and I'm a 3D Environment Artist. I've been interested in both art and video games since I was a child. I studied Interactive Design at university and later discovered 3D modeling while studying in Japan. Over time, my interests and career direction gradually shifted toward 3D art, which eventually led me into the game industry.

Influenced by Japanese animation and Studio Ghibli films, I've always been drawn to hand-painted and stylized visuals. I'm also passionate about using shaders and procedural texturing workflows to achieve a hand-painted look. To me, building visuals through nodes and procedural techniques feels a little like magic, and I'd love to keep exploring the possibilities of combining procedural techniques with hand-painted aesthetics.

Inspiration and Goals

The scene was inspired by an illustration by Japanese artist Nonohara. One evening, while looking through a collection of Nonohara's illustrations, I came across this piece and was immediately drawn to its warm and whimsical atmosphere.

I started wondering what it would feel like to actually step into this scene, and that became the starting point for the project. In Unreal Engine, I wanted to reinterpret the atmosphere and painterly quality of the original artwork as an explorable 3D environment, while also adding my own ideas and imagination.

During the project, I also experimented with different seasons and times of day. I used it as an opportunity to practice Substance 3D Designer and procedural texturing workflows, exploring how procedural techniques could be used to achieve a hand-painted look.

Concepts & References

I think gathering references is a very important step at the beginning of a project. It not only helps shape the direction of the work, but can also lead to unexpected ideas and inspiration. For this project, I collected a large number of image references in PureRef. Since my goal was to stay as close as possible to the design of the original illustration, I didn't spend too much time looking for separate references for individual assets.

Instead, I focused mainly on atmosphere and lighting. I spent a lot of time analyzing the lighting in the original artwork and gathering references with a similar mood, hoping to capture the feeling of the original piece as accurately as possible.

Tools & Workflow

Most of the assets were created in Maya. For assets that required sculpting, I first created the base meshes in Maya and then brought them into ZBrush, where I focused on refining the major forms, silhouettes, and stylized surface details. After that, I used Substance 3D Painter for baking and texturing.

The trees and bushes were mainly created in SpeedTree, while the leaf masks were painted in Photoshop to define the foliage shapes. The final stylized look was mainly achieved through material shaders in Unreal Engine.

The blockout was mainly created in Unreal Engine. I started by setting up the camera based on the original illustration and tried to match its perspective as closely as possible. From there, I used simple geometry to quickly establish the major shapes of the environment. I then adjusted the scale and placement of the buildings, trees, and other major elements while checking the overall composition through the camera.

I also added some simple lighting to evaluate the light, shadow, and atmosphere. Throughout this process, I gradually replaced the blockout geometry with finished assets and continued refining the scene until it approached the final look I wanted. 

Improving Efficiency

To improve efficiency, I didn't follow the same traditional workflow for every asset. For example, the brick walls and roof tiles were mainly created using Nanite and Displacement Materials, allowing me to skip the additional sculpting and baking process.

For the snow coverage, I created a PCG Snow Coverage setup in Unreal Engine combined with a Triplanar Mapping snow shader. This allowed me to achieve the sparkling, magical look I wanted while avoiding the extra work of creating separate snow meshes and UVs.

Texturing Workflow

Most of the textures in this project were created in Substance 3D Designer, and one of my main goals was to explore how procedural texturing could be used to achieve a hand-painted look. In Substance 3D Designer, I found that one of the key techniques for creating a hand-painted feel was applying an appropriate amount of Slope Blur to both the Base Color and Normal.

For the Base Color, I used Slope Blur to break up edges that would otherwise look too smooth and regular, creating more irregular shapes and watercolor-like marks. I applied a similar Slope Blur treatment to the Normal to introduce subtle irregularities to the surface.

Under lighting from different angles, these variations create brush-like surface details that further reinforce the painterly feel.
Finally, in Unreal Engine, I fine-tuned the Tiling, UV Scale, and color contrast through the materials to make the different textures and objects feel more cohesive and harmonious within the scene.

Vegetation Shader

Here, I'd like to highlight the material shader I created for the foliage and bushes. In addition to using Spherical Normals, which are commonly used for stylized vegetation, I created a stylized foliage material to reproduce the distinct color blocking and light-and-shadow relationships found in traditional hand-painted artwork.

Hand-painted vegetation usually doesn't rely entirely on realistic lighting. Instead, volume is often created through different color blocks, gradients, and localized highlights. I broke this approach down into several controllable shader modules, using Gradient, Bottom Shadow, Front Color, and Top Highlight to recreate these hand-painted lighting and color layers in real time.

I also created a camera-distance-based self-shadow control, allowing the shadow detail on the foliage to change depending on the viewing distance. More volume and detail are preserved up close, while the effect becomes simpler at a distance, helping the overall scene maintain a cleaner and more consistent hand-painted look.

Water Shader

For the water material, I also wanted to maintain the overall hand-painted style rather than aiming for a fully realistic water surface. I used a combination of different colors and noise patterns to create variation and a sense of movement in the water. I also added an interactive effect between the water and surrounding objects.

When rocks or other objects intersect with the water surface, ripples and foam are automatically generated around the contact areas. This helps the objects blend more naturally into the water while adding more movement and reinforcing the painterly feel of the scene.

Assembling the Final Scene

For the final scene, I used Niagara to create particle effects that added subtle movement and atmosphere to different weather conditions and times of day.

To reinforce the painterly look, I used small grass cards combined with an RVT Material for the thatched roof, allowing them to blend more naturally with the roof surface while maintaining the volume of the thatch. I also used an Overlay Outline Material on selected assets to add outlines that mimic the linework and brushstrokes of the original artwork.

For the composition, I used the original artwork as a foundation and planned several key camera angles in advance. As I refined the scene, I continuously balanced the architecture, vegetation, and terrain. I also used subtle color variations for trees and rocks, self-shadow control, decals, and scattered small rocks to reduce repetition and strengthen the sense of depth between the foreground, midground, and background.

For the lighting, I used Lumen, focusing mainly on the light direction, the balance of light and shadow, and fog. For different weather conditions and times of day, I combined Exponential Height Fog with fog cards to create different moods. Most of the final color adjustments were done in the Post Process Volume, with additional Bloom used for the nighttime scenes.

The camera animation was created in Unreal Engine, and the final sequences were rendered using Movie Render Queue. 

Conclusion

The project took around three months to complete. One of the biggest challenges was maintaining a consistent world and art style across different seasons and times of day. Although the lighting, colors, and atmosphere could change significantly, I wanted every version to still feel like it belonged to the same world.

Another challenge was adjusting the layout and composition while staying true to the original artwork. Since a 2D illustration is very different from a 3D environment that can be viewed from multiple angles, I had to continuously balance the placement and scale of elements such as the river, terrain, vegetation, and architecture, making sure they felt natural and well-balanced from different camera angles while still preserving the atmosphere of the original piece.

I also found that after working on the same project for a long time, it was easy to lose objectivity. Sometimes I could feel that something wasn't quite right, but couldn't immediately identify the problem. To deal with this, I would periodically leave myself some feedback and revisit the project later with fresh eyes. When I came back to it, I could often spot issues that I had previously overlooked. Learning to step away from my work from time to time was also an important lesson from this project.

Advice for Beginner Artists

If I could give some advice to artists who are just starting out in 3D, I would say stay curious, observe as much as possible, and don't be afraid to experiment with different approaches. Try not to become too attached to one specific workflow — tools and techniques are ultimately there to help us express our ideas.

I also think it's very important to finish your projects. You will inevitably run into problems along the way, and sometimes you may even lose perspective on your own work, but that's all part of the learning process. Taking a break and coming back to your work with fresh eyes can often reveal things you didn't notice before.

Most importantly, find what you genuinely enjoy and keep creating. Rather than focusing only on technical complexity, I think developing your own artistic taste and way of expressing ideas is much more important in the long run.

Shuming Zhang, 3D Environment Artist

Interview conducted by Stephanie Almogabar