The Photon Water System: Developing a Next-Gen Water Rendering Solution for Games

Introduction

Zhenyu Mao: I am a member of the tech team at LIGHTSPEED STUDIOS, working in collaboration with experienced professionals from the gaming industry and senior researchers specialized in computer graphics and simulation. Before joining LIGHTSPEED, I had more than 20 years of experience as a 3D programmer and tech lead at Ubisoft Shanghai and Ubisoft Toronto, where I made major contributions to the development of several AAA titles, such as the Splinter Cell series, and Far Cry 5 and 6.

I am happy to be part of the current team at LIGHTSPEED, where I am tasked with integrating and implementing the latest cutting-edge technologies into our games. It is truly inspiring to collaborate with such an incredibly talented group of professionals.

Kui Wu: I am a senior researcher in the Tech Center at LIGHTSPEED STUDIOS. Before joining the team, I was a postdoctoral associate at MIT CSAIL. In 2019, I received my Ph.D. in Computer Graphics from the University of Utah. As a researcher at the Tech Center, I am working on a variety of physics simulation features, including fluid, hair, cloth, etc. We are developing both offline and real-time simulation techniques based on requests from our current game projects and to target the next-gen games.

Developing a New Solution for Water Rendering and Real-Time Simulations

Water rendering is one of the most essential and required features within LIGHTSPEED STUDIOS. The current water solution in Unreal Engine is not suitable for production, and third-party plugins suffer from performance issues and do not scale up to large worlds.

We decided to tackle the issue head-on by developing a new water rendering solution that would meet the needs of LIGHTSPEED STUDIOS. We leveraged cutting-edge technologies to achieve realistic water rendering and simulation, while also ensuring optimal performance and scalability. This enabled us to provide a production-ready solution for their needs.

Photon Water System

Zhenyu Mao: I began working on the Photon Water System as the sole developer two years ago. The initial idea focused on creating a pipeline for rendering water, using splines to generate rivers, FBM waves to generate detailed waves, and adaptive tessellation to render large bodies of water. However, with Dr. Kui Wu joining our team – who has a strong background in physics simulation – we were able to take our water technology to the next level.

Thanks to Dr. Wu's expertise, we have developed a proprietary solution that generates natural water flow that interacts harmoniously with the environment. Our water simulation can generate accurate flow around objects, creating realistic turbulence and generating foam in the appropriate places. 

Kui Wu: I joined the project half a year after Zhenyu. I was looking for a way to apply a physical-based simulation technique to the game. Real-time water simulation seemed like an ideal choice as it is ubiquitous, ranging from small-scale things like a cup of water to large-scale bodies such as rivers and oceans. There are many papers that discuss real-time fluid, but there are few games that feature large-scale real-time water simulations. We thought this would be an important feature for the gameplay.

The Production Process

The production process for Photon Water is designed to be straightforward. Artists can use spline tools to create rivers and lakes or generate water procedurally using external tools such as Houdini. Alternatively, they can assign the water material to any mesh. Once the desired water sources have been added, the fluid simulation can be initiated with a single click. The simulation results are saved as water height and flow maps, which are stored as tile-based virtual textures. At runtime, these virtual textures are loaded on demand.

Photon Water System provides physics support for gameplay programmers, enabling them to obtain water wave height and flow vector from the API for swimming or boat physics. We harness the Unreal Engine's single-layer water material to obtain Lumen lighting and reflections. Moreover, we have also developed a physically-based underwater volumetric lighting system.

One of the primary obstacles we encountered while developing Photon Water was guaranteeing that the simulation results were uniform across all projects. While the system operated perfectly on our trial map, it would occasionally fail in other settings due to algorithmic constraints. To address this challenge, we had to pinpoint all the extreme scenarios and add safeguards to guarantee that the simulation functioned dependably in all contexts.

A Combination of Lattice Boltzmann Methods and Shallow Water Equations

Kui Wu: Inspired by existing water plugins and demos, we first experimented with LBMSWE (Lattice Boltzmann Methods for Shallow Water Equations). It is highly parallelizable and supports turbulent flow with a high Reynold number, making it great for flow map baking.

However, it was found to be unsuitable for real-time water simulation for two reasons. Firstly, due to the nature of LBM, each grid cell needs to store three distributions, each with nine channels, leading to a great memory cost. Additionally, the LBMSWE framework is still a single-phase simulation technique that cannot accurately capture the free surface of the water. As such, we only used LBMSWE to pre-compute water data.

For runtime water fluid, we follow the traditional Shallow Water Equation (SWE) to solve the water dynamics on a 2D height field grid, where the height indicates water height. Pure SWE is extremely cost-effective to compute and has a low memory footprint.

Moreover, we have developed a grid-based foam simulation method within the same framework. Thus, the water foam can be generated and propagated based on the velocity field solved by SWE, increasing the realism of the water system.

Zhenyu Mao: We modified the water rendering system to adapt to the runtime-generated data. Height maps and flow maps are dynamically updated, cut into tiles, and written into the virtual texture pool directly. The water quadtree is also updated at runtime from the GPU, allowing the rendering code to work with both pre-computed and runtime data.

Furthermore, we developed a third type of simulation – surface wave simulation – which only solves the pressure function in the shallow water equations. This allows us to create interactive waves when players or objects interact with the water, applying a displacement on the existing water surface. This simulation runs quickly and uses only two depth render targets, making it possible to work even on mobile platforms.

The Photon Water System provides various parameters to control the simulation, including grid size, time step, viscosity, gravity, friction, relaxation factor, inlet/outlet speed, and CDLOD tessellation factor for rendering. In most cases, the default parameters work well out of the box.

One of the challenges we encountered during development was ensuring that the simulation result was consistent across different projects and environments. To address this, we identified, fixed, and protected against various edge cases to ensure the simulation would work reliably in all environments.

Scalability and Efficiency of the Photon Water System on Different Platforms

The solution is designed to be scalable across different platforms, from consoles/PC to mobile. The runtime SWE simulation runs efficiently at a simulation domain of 512x512, using five persistent buffers of the same size and a few temporary buffers. It only takes 0.5 ms for all the simulation steps on the PS5.

The surface wave simulation is taking less than 0.1ms on the PS5, and it has been tested on mobile devices, proving to be able to run efficiently in 1 ms with a resolution of 512x512 on a device with a Snapdragon 865 chipset.

Regarding rendering, the water render takes approximately 2 ms in total when the water takes up a majority of the screen, with approximately half of the time spent on the lumen reflection. 

Future Plans for the Photon Water System

Looking to the future, we plan to experiment with several exciting features. We are exploring the possibility of increasing the simulation resolution, which is currently set to one square meter per pixel. This means that objects smaller than this size are ignored. By increasing the resolution, we can create more detailed simulations. Additionally, we need more optimizations to ensure that the performance budget is kept under control.

We are also interested in exploring the potential of momentum transfer between pre-computed water data and runtime data. We have tested dampening and blending the two types of data at the border, but we believe there is potential to connect the two seamlessly for even more realistic simulations.

Zhenyu Mao, Principal Software Engineer at LIGHTSPEED STUDIOS

Kui Wu, Senior Research Scientist at LIGHTSPEED STUDIOS

Interview conducted by Arti Burton