Ubisoft on Building Efficient Player Bots Using Deep Reinforcement Learning

Ubisoft's Research and Development Scientist Joshua Romoff has revealed some of its AI achievements in a talk called "Deep Reinforcement Learning for Efficient Player Bots." He showed how player bots can be made more engaging and efficient, continuing the topic of creating more realistic NPCs brought in 2022.

Romoff mentioned that traditional AI methods sometimes can't handle complex and dynamic game environments, but DRL "offers a way to train bots to navigate with player controls, allowing for more believable navigation patterns and a better experience for the player."

Moreover, it can be used to specify particular behaviors so developers can train bots to perform specific tasks, like respecting speed limits and staying on a certain path in the case of self-driving cars.

Another great example was demonstrated on Ubisoft's For Honor. What makes it interesting is that bots have to take into consideration their opponent's moves in order to win, which is more challenging than in previous instances. 

"Building a bot that can't be easily exploited by players is not a simple task. To accomplish this, traditional methods typically cheat – by knowing exactly what the player is about to do and countering it." So the studio trained its AI, the DeepBot, using competitive self-play, where the agent continuously learns based on previous versions of itself. This made it pretty hard to beat.

Ubisoft's DRL achievements are advanced, so if you're interested in seeing how it works, I suggest this awesome video by AI Warehouse, who made AI teach itself to walk.

Find more information about Ubisoft's DRL ventures here and join our 80 Level Talent platform and our Telegram channel, follow us on Instagram, Twitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.