Jan Hlousek on Setting Up In-Game Physics & Optimization

Introduction

80.lv: Please introduce yourself to our readers.

Jan Hlousek: Hey, my name is Jan Hlousek and I am the Technical Director at Keen Software House, a Czech developer of Space Engineers and Medieval Engineers. Currently, I am overseeing the development of the 3.0 version of our in-house engine, which will be utilized in our upcoming games.

My past game development experience reaches to 1996 when I established the game development studio Centauri Production with our most notable titles adventure series Memento Mori and zombie shooter Pound of Ground. I also worked in mobile game studios Disney Mobile and Geewa.

Working on Space Engineers

80.lv: What were your main tasks when working on Space Engineers? What bottlenecks did you have to deal with?

Jan Hlousek: Initially, I joined Keen as a Render Engineer responsible for finalizing the adoption of DirectX 11 with PBR workflows, adding GPU particles, post processes, and many other render-related tasks. Once released, I moved on to improving multiplayer, where the biggest achievement was minimizing the lag using client-side prediction, which is quite challenging given the physics-based nature of the game. Next, I ported the game to the Xbox console, while slowly moving to a more management role, overseeing the PlayStation port, physics overhaul, and the new engine development.

In-Game Physics

80.lv: Let’s say there’s a huge spaceship with smaller ships inside and they’re also characters running and jumping around, how do you make sure that the game's physics can handle such a task?

Jan Hlousek: Interestingly enough, this is not as challenging as it sounds, since all those objects are RigidBodies in the physics engine, thus integrating and colliding gracefully. The challenging part is that the bodies consist of many blocks. Players are building ships out of these blocks, so these bodies can change shape anytime – either by players placing or removing blocks or by ships colliding and blocks being deformed and destroyed. Although we are using Havok, we had to heavily customize it with our own collision shapes to deliver the target performance.

80.lv: Making a complex system with physics work is a challenge but implementing the scenario in a multiplayer game is a different level. Could you please explain the challenges that appear when you try to launch such systems online?

Jan Hlousek: Multiplayer is an entirely different beast in this scenario and indeed proves quite challenging. If all players have perfect internet conditions, we could just stream position updates from the server and everything would be fine. Clearly, this is not the case, we have players with latency up to 500ms, which renders the game unplayable: you would press a button and see the outcome in half a second, which can be quite confusing.

The usual solution to this is predicting the outcome of the inputs on the client, displaying the results and just correcting once the server delivers its ground truth. In our case, it gets complicated by players usually inhabiting moving platforms or ships, which are piloted by other players. There is no way you can predict the outcome of another player's ship movement, which affects you. So the solution is hierarchical prediction. It sounds simple, but there are many caveats. For anyone interested, I would recommend my talk from GDC 2023 about our approach, called "Predicted Physics-Based Multiplayer in 'Space Engineers'".

Optimization

80.lv: When there’s no sync between the client and the game server, how can you minimize such situations on the tech side and optimize synchronization avoiding bugs?

Jan Hlousek: The main prerequisite to reducing client/server desyncs is having deterministic simulation, having the same outcome on client and server. In multiplayer though, there are unforeseen situations given there is always an older state on the client than on a server. In that case, correction will happen once the new data arrives. Of course, corrections have to be applied subtly and over time, not to be distracting for players.

Conclusion

80.lv: What are the rules that help minimize uncertainty in multiplayer games? How can you tweak servers and provide the best possible experience? Could you share some tips?

Jan Hlousek: In catastrophic scenarios (too many corrections have to be applied), we just switch off the client-side prediction completely. That will make sure everything is in sync, but also means the latency hits hard.

There are a few rules to keep in mind:

• Have your server on a good connection with a provider being close to the backbone.
• Indicate the server location to your players in the server name or world name.
• Educate players with relevant information, like the ping time displayed in the join screen when selecting a server to connect to, this value should be as small as possible.

Jan Hlousek, Technical Director

Interview conducted by Arti Burton