The secret is in Deep Reinforcement Learning.
Ubisoft La Forge – the company's R&D initiative – is working tirelessly to "bridge the gap between academic research and videogame innovations" and use the latest technologies in Ubisoft's videogame production.
Joshua Romoff, a data scientist at Ubisoft La Forge, explained how deep reinforcement learning helps create more realistic NPCs.
Deep reinforcement learning is a type of machine learning that uses AI to find the most efficient solutions to a variety of problems. Romoff and his team try to encourage or discourage certain kinds of AI behavior with rewards and penalties: giving it points for doing something they like or taking points away for something they don’t. The goal of the AI is to get the highest score it can with the actions available.
At the same time, a deep neural network "takes the screen, processes it into something at a much smaller dimension, analyzes the data, and then inputs that information into the reinforcement-learning part, which then performs actions based on that input data."
Romoff shared that the team did some testing in Hyper Scape – Ubisoft's free-to-play first-person shooter battle royale game – to create more “player-like” bots using reinforcement learning. The game's map is full of areas where you can't get by just walking, so throwing the bot in a training loop so it would learn how to navigate the map using jump pads and double jumps "made sense."
Such kinds of tests are useful to see the AI's optimal behavior:
"You might notice these things it’s learning and figure out that the behavior is actually helping the agent achieve its goal, which could point to something you weren’t aware of, allowing you to debug and know if your code is working as expected."
The scientist believes that the latest generation of consoles and cloud and streaming services helped develop Ai in games:
"We’re at the point where we could solve these problems, and have things like bots that navigate really complicated maps in 3D worlds with all these crazy abilities. Now it can run fairly efficiently, and act much more human-like than anything we could just hardcode."
Reinforcement learning can be used not only on bots but also for server management or optimizing sequential decision-making to get AI to look at the number of players at certain times of day and increase or decrease the current number of servers online at a given moment based on needs.
Romoff is planning to keep working on injecting more realism into games, making things like NPCs and bots feel more human, and solving problems that haven’t been possible until now. He also wants to get game designers to use the tech and create player-facing tools with it.
Check out the interview here if you want to learn more and don't forget to join our new Reddit page, our new Telegram channel, follow us on Instagram and Twitter, where we are sharing breakdowns, the latest news, awesome artworks, and more.
Keep reading
You may find this article interesting