Creating Volumetric Clouds for Mobile in Unreal Engine 5

Introduction

My name is Erfan Ghaemian, I’m a Technical Artist at North Wind, an independent game studio where we develop mobile games. Today I want to talk about how I managed to create my custom volumetric clouds shader in Unreal Engine 5 for mobile.

The Volumetric Clouds Project

I started this side project exactly a year ago with the mindset that I might not be able to pull this off at all as it was my first time doing something like this, but there is also the fact that creating a volumetric clouds system in 2022-2023 is much easier than, let’s say, in 2015-2016 as more people have attempted this and shared their valuable knowledge in the process. Most of my code is based on Fredrik Häggström's Real-time rendering of volumetric clouds.

What Is the Challenge?

Unreal Engine has its own volumetric clouds system, but the problem is that it does not support mobile platforms, which is understandable because the techniques used to achieve the effect are very demanding for mobile hardware or outright not possible.

Switching from the default preview to the Android preview in UE5.1 removes the clouds.

What Is the Target Hardware?

The target hardware is my smartphone Sony Xperia XZ Premium equipped with Adreno 540 GPU released in 2017. For people not familiar with mobile chipsets, I would say Adreno 540 is for mobile phones what NVIDIA GeForce GTX 1080 is for PCs – old but capable!

For curiosity's sake let’s compare a smartphone GPU with a discrete PC GPU.

According to UploadVR, Adreno 540 gets ripped to shreds by a mere 1060. Now I know this is not a fair comparison as the architecture and the OS are completely different, nor the device I mentioned is the same, but you get the point.

What Is the Technique?

The fundamental technique used to achieve this effect is ray marching. In simple terms, we cast a ray from the camera view. Along the ray, we sample the volume texture at set intervals and then the textures get accumulated along the view and give us a nice volumetric look. The number of times we sample the cloud texture is called the number of steps and the distance in between is referred to as step size. Now, obviously higher step count means higher performance cost simply because the GPU will have to render the defined amount of steps for every pixel covered on screen by the ray marcher.

The red dots are the steps and the space between them is the step size.

Notice that in the above image, the steps start once inside the box mesh. That’s because, for more efficiency, we use a method known as ray box intersection, which gives us better control of our ray marching.

Creating this in a custom node would have been easier, but for maximum optimization, it's recommended by Epic Games to keep it outside of the custom nodes as much as possible.

For clouds to sort accurately when used with other objects in the scene, we need to disable the Depth Test inside our material settings and, as you may have noticed in the above nodes, we are already using a Scene Depth node inside our ray box intersection that fixes the sorting problem. 

Also, to move through the clouds we need to use a box mesh with inverted faces. This can simply be done in Maya or Blender.

A box mesh with inverted faces created in Maya.

What Are Volume Textures?

Volume textures, also called 3D textures, are a set of small 2D textures put together where layering each slice behind another would give us the 3D volume look.

A basic material setup demonstrating how a volume texture can be used.

GIF created by Ryan Brucks from Epic Games.

How Do We Create Volume Textures?

The fastest and easiest way would be to use Epic’s tools which you can access by enabling the Volumetrics plugin inside Unreal Engine. Then, by searching BP_Draw_Tiling_volume you can create your own volume textures easily. If you want to learn how to create your own tools, I suggest this article by Asher Zhu.

How Does It Work?

We use a 2D texture called WeatherMap that enables us to define which parts of the map are to be covered by the clouds at what level of density and height.

Cloud-shaped UE logos with the left one being at full density and height; the middle one has a gradient height and the right one has a gradient density.

To define the base shape of our clouds, we use a volume texture called Base Noise.

Tiling of Base Noise texture.

And to add further detail, we use another volume texture named Detail Noise.

Tiling of the Detail Noise texture.

Now let’s set our step count to 64 and export the current state of our clouds to see how it performs on mobile without any optimization.

Each second it takes 300ms to render a frame, thus three FPS.

As expected, we barely got three FPS with the current setup, and it has only half of the screen covered. So let’s go for some optimization.

1. We must tell our ray marcher to stop rendering if it reaches full opacity before the loop has ended. For example, let’s say that by step 16, the density of our accumulated clouds has reached full opacity, then our max step count is 64. So rendering beyond that is a waste of performance.
2. Our ray marcher should stop rendering parts of the textures with an opacity of 0, but keep in mind that once we start factoring the light calculation later, this will cause some artifacts.
3. We need to increase the Mip/LOD of our textures as clouds are farther away from our main camera.
4. As clouds are farther away from the camera, the step count should go much lower.
5. We will also stop rendering the clouds after a certain distance. But this might look off in the distance, so we need to tweak our background color carefully, therefore it blends with our clouds.
6. By disabling Apply Fogging in the material settings, we get an acceptable reduction in shader instruction count.

After applying these to our code, let's test again!

Without changing the Mip level the performance gain is not that great.

Increasing the Mip level affects the detail but at a considerable performance gain.

That is a massive boost, reducing render time from 300ms to 40ms! But that’s still not enough when you consider that the only thing available in the scene are clouds.

But hey! We still have other tricks up our sleeves for optimizing it even further. But let’s continue lighting our clouds as we will return to this again.    

In order to calculate the lighting for our clouds, we need to do another ray marching toward the sun for every density step. This means if we have 64 density steps and 4 light steps for every density step, that would be a total of 256 steps! This can easily get out of control, so we have to be very careful with it.

Ray marching towards the sun for every density step.

Lighting gives the needed depth to our clouds.

By adding the lighting, we got a whole new look! Now after applying the lighting calculation, we can guess that this will run terribly, so let’s go for another round of optimization.

Main Optimization

The most effective method for optimizing clouds is to render them in a lower resolution. To achieve this in Unreal Engine, we need to use a SceneCapture2D component to store only volumetric cloud data in a texture called a Render Target so that we can easily adjust the resolution and access it.

Let’s export it to our device again and see how it goes.

We are getting a solid 60 FPS even when most of the screen is covered by clouds.

We now get 60 FPS with 64 density steps and two light steps with volumetric cloud render target resolution at 320x180 and device resolution at 1920x1080.

When viewed on a large screen, the image appears blurry, but on a smartphone screen, this issue is less noticeable. However, the artifacts resulting from our previous optimizations are still visible. To address this, we can apply a Box Blur, which is both inexpensive and highly effective at mitigating these artifacts.

Box Blur with only four samples cleans up the artifacts at a minimum cost.

Box Blur code credit goes to TechArtAid.

At this point, we have managed to get 60 FPS. But two major issues need to be addressed.

1. When viewed from certain angles, we can see sliced artifacts in our clouds that are unpleasant to look at. To fix this, one way would be to increase the step count extremely, but that would come at a performance cost. The other method would be to use a noise texture to offset the steps. Ideally, blue noise is recommended, but I prefer Dither Temporal AA. But unfortunately, this method does not perform well on mobile hardware. What makes it worse is that since we are using a low-resolution render target it results in worse image quality.

Dither Temporal AA node cleans those artifacts nicely at a low cost on PC.

2. By storing our clouds in a render target, we lose the depth info. It means the clouds will render over everything and make It unusable. So the solution I came up with is to create another cheap ray marcher that will get the depth info and multiply its result with the final render target to make it look the way it should. The drawback of this method is that the devices with older architectures cannot render the scene depth properly. For reference, my Adreno 540 could not render the depth properly, so I tested it on an Adreno 612, a lot weaker than what I had, but it renders the depth properly.

Conclusion

Though there are some things left to do, I think it’s enough up to this point. In the future, maybe I will come back to add stuff like shadow casting, crepuscular rays, or multiple layers of clouds.

Thank you for reading, I hope you have found it useful!

If you have any questions you can contact me by email.

Here are some useful links for those that want to explore in more detail:

Understanding the basics:
Volumetric Ray Marching Cloud Shader by dmeville

Neat tips and tricks:
Creating a Volumetric Ray Marcher by Ryan Brucks

For better technical understanding:
With Your Head in the Clouds by Julian Oberbeck

The creation process of Horizon Zero Dawn Volumetric Clouds:
The Real-Time Volumetric Cloudscapes of Horizon Zero Dawn by Andrew Schneider

My main source code and Best in detail:
Real-time rendering of Volumetric Clouds by Fredrik Haggstrom

Free to download as a good reference:
Volumetric Clouds by Harry Emelianov

Erfan Ghaemian, Technical Artist