Creating Animated Painterly Forest Using SpeedTree, Substance 3D, & UE5

Introduction

My name is Anastasia Kasyanik, and I’m a 3D Environment Artist based in France. I studied Game Design in Paris, where I quickly discovered the world of 3D thanks to my school. After many late nights finishing student projects, earning my degree, and doing my first interview on 80 Level, I finished rebuilding my whole portfolio from scratch and found my first job. Since then, I’ve contributed to several game productions such as Orphan Age, Loftia, and No Rest for the Wicked, and I also taught 3D for a short time at the school where I studied.

While working on both personal and professional projects, I soon realized that stylized art inspired me the most. Today, I enjoy creating environments, especially natural landscapes, where I try to tell small and subtle stories.

Unfinished Personal Project

Getting started

I began this project around 2021 in Unreal Engine 4. The idea was to make a short animated video, showing a collection of natural environments someone might wander through, where the shots would follow one another to the beat of dark ambient music. Something that would evoke loneliness, nostalgia, and a slightly mysterious mood, evoking something like a soft dreamcore feeling, far away from everything, drifting from reality, yet still connected to the real world in some way. 

I wanted to share this feeling based on a childhood memory of the western forest landscapes near where I used to live in eastern France. This series was supposed to showcase lots of mountain environments that I never made.

To find an art style that could work well in 3D, I looked for references that matched what I had in mind. I drew inspiration from concept arts, Ghibli animations, paintings, photography, nature itself, and other 3D environments. I usually like to start with a big Pinterest search session. Here are a few of the main inspirations I remember using:

References helped me spark ideas, but I never knew in advance what compositions I’d do beforehand. I like to rely on intuition. My workflow usually starts with creating as many props as possible before set dressing my scenes.

Production Workflow

Vegetation

I generally like to start with grass modeling so I can quickly cover large areas before moving forward. There are many ways to make 3D grass, and I chose to fully model it, with each blade as a mesh with no texture or alpha. I created a few versions of grass clumps so I could vary their size and shape to avoid a flat and uniform look of grass fields.

It doesn’t need to be perfect. The individual shapes won’t be noticeable once thousands of blades cover the ground. It only needs to feel fluffy enough.

I explored many different ways to render grass in Unreal Engine and did many tests during this project. The first versions were made with a simple grass shader that included options like color variation driven by tiling noise textures, separate colors for the tips and the base, wind color, and subsurface. Later on, I discovered the Runtime Virtual Texture system, which lets the grass blend perfectly with the ground, but that was something I only started using afterward.

I also tried different versions of grass fields, with shadows on and off. Both looked good for a stylized project, but making it shadowless is better for performance. And to ensure a smooth look, I also had to put all the normals of my grass meshes facing up. Without this step, it didn’t look good at all.

After the grass, I moved on to the plants. This time, I decided to make them with textures that I hand-painted in Photoshop and used opacity masks.

PrismaticaDev’s videos also helped me understand Unreal Engine tools better, especially vegetation optimization and shader tricks. I learned a lot thanks to his great explanations.

Trees

My main inspiration for the trees came from Dragos Matkovski’s The Illustrated Nature project. I loved how fluffy and painterly his trees looked. While searching for a similar approach, I found Pontus Karlsson’s “fluffy stylized trees” video on YouTube. Huge kudos to him for making this tutorial. This video started a long journey of experimentation and constant back-and-forth between Blender and Unreal Engine for me.

I tried many approaches for the modeling part. I eventually chose SpeedTree for the trunks because it allowed me to create variations quickly. Then I used Blender’s Smart UV Project to unwrap them, and finally textured everything with a mix of smart materials and hand-painted techniques in Substance 3D Painter. This workflow helped me produce many game-ready assets efficiently.

For the foliage, I mostly followed Pontus’s method. I also opened up the bottoms of the blobs so they would feel more natural and avoid overly dense shapes.

Another simple trick that worked well was for the UVs. Pontus’s idea was to fill each quad of the mesh to the entire UV space (in Blender, you can do so with U > Reset) so one leaf texture can drive the shader. But I wanted the result to look messier and chaotic. So I added an extra step and randomly selected groups of UV quads of my modeled foliage in Blender and rotated them.

Pontus’s tutorial was originally for Unity, but he provided a UE4 setup here. I still adapted the shader because I wanted to add more options.

For example, I didn’t use a camera-facing setup. I preferred the leaves to stay still rather than making them rotate based on the camera. For the World Position Offset input, I forgot to set the first TransformVector to a Local Space destination, but the result looked better, more chaotic, so I kept it.

I also added nodes to the Normal material input to smooth the overall look. I noticed it made the shader foliage look smoother with this setup. However, it required some normals orientation tweaking, and I played a lot with those in Blender to have a smooth look for each tree type.

Another option I added was a foliage opacity control. This allowed me to indirectly adjust the shadow intensity. To avoid uniform-looking trees from afar, I also added a color variation parameter using a tiling noise texture. The noise color could be chosen separately from the main one. Later, I connected a subsurface parameter, which made the foliage look smoother, but it influenced the main color, so I added a second node for a separate top color control.

Rocks

To bring some variation while staying in a natural theme, I modeled some rocks as well. I created a simple shader that adds moss only to the upper faces. With a few rock models, I could rotate them freely and get lots of variation thanks to the moss shifting depending on each rock’s orientation.

Set Dressing

This is my favorite part. Once the assets were ready, I began creating the environment compositions. As I mentioned earlier, I didn’t start from a specific concept; I followed my intuition, so I didn’t make any blockouts. In Unreal Engine, I love using the Landscape tool to quickly sculpt terrain and the Foliage Mode to populate it. It’s simple, gives a lot of control, and saves time compared to sculpting terrain in external software and placing props one by one. This workflow allowed me to build large environments rather quickly. Here I made a time-lapse video on YouTube of the full set dressing process of an environment that’s supposed to be in the same collection as my unfinished project:

Lighting

I didn’t focus much on lighting for this project. I mostly used Unreal Engine’s default sun by adjusting its rotation and intensity while set dressing. I occasionally added a few god ray sheets using simple textures with alpha on planes.

Distance Fog & Post-Process

For static environments with no free-camera movements, I like to add fake distance fog. It’s cheaper, easier to set up, and visually effective. I used planes with a base color and opacity, and placed several of them based on the camera view to create a fading effect in the distance.

At the same time, I used a basic Post-Process Volume to manage final touches, like saturation, contrast, shadows, and midtones. I tweaked all these parameters until the colors felt right.

Wind

The wind effect I used is based on PrismaticaDev’s tutorial video “Simple Global Wind in UE4”. It can be easily managed from a Material Parameter Collection, which allows me to have a global wind control across all plants, grass, and trees at the same time.

Conclusion 

As I put in the title, this project is unfinished. If I recently decided to publish in my portfolio the few experimental landscapes I made, it’s because I realized I probably won’t work on it anymore. It has been dragging on for too long, has become a bit messy, and my inspiration has faded. I’d rather start something new with fresh ideas. 

Still, even unfinished, this project was far from useless. I had a lot of fun working on it, and I was happy to have time to explore all the ideas that inspired me. It was my first large-scale project, and it took me a while to get started because I wasn’t confident with big environments yet. As a beginner, I never knew where to start or which workflow to follow. This project taught me a lot about creating natural environments at every stage of production. It also felt a bit nostalgic to dive back into this old project while writing this interview.

If you’re interested in the tree art style I used, I’ve put together a small UE5 scene with several trees and their foliage shader. You can download it from my GitHub profile here.

It’s free, and you can use it for whatever project you want. Just be creative!

Anastasia Kasyanik, 3D Environment Artist