modl.ai CEO on ML Bots in Games & The Company's Al Engine

Introduction

80.lv: Could you please introduce yourself and modl.ai to those who still don't know you?

Christoffer Holmgård: My name is Christoffer Holmgård, and I'm a Co-Founder and the CEO of modl.ai. In a nutshell, at modl.ai, we create AI for games and, specifically, bots for games for different purposes. We do all that under something that we call an AI engine. modl.AI is all about showing the game development community that there is something called an AI engine that they can use for Al and games. For us, that means driving bots inside of games to take over a player's character, for example.

I started out many years ago being very interested in games, as many people are, but when it was time to get an education, I started out in Psychology and thought I'd leave games behind—at least for the most part. Then, I studied psychology but ended up becoming very interested in cognitive psychology and psychometrics, which is sort of like behavioral modeling and simulations. So, in many ways, I found a way to turn psychology into something that's super relevant for game development. I also got very interested in statistics and those kinds of things. All these areas lead me to artificial intelligence in the long run.

I spent a bunch of years working as an indie game developer. I shipped titles for Flash, back in the day when that was a thing, and for different consoles — PlayStation, Switch, and Xbox. I ran a game studio, but at the same time, continued to do research and psychometrics, which eventually turned into AI and machine learning. Then modl.ai came together when I joined the research community in Al for games and then, after a few years, having met all the people behind modl.ai. We started modl.ai in 2018 as a company whose purpose is to bring AI and machine learning development tools to as many game developers as possible. 

ML Bots in Games

80.lv: In the last couple of years, what were the biggest changes in the ways that machine learning bots worked in games? 

Christoffer Holmgård: I would say that a lot of this, when you're looking at it from behind the scenes or if you've been part of the AI research community, you know it's been a process that's been coming since 2014 or so, because that's when people started doing deep learning. That is when people figured out that we can use things, such as graphics cards and GPUs or some of those principles to scale what you can do with machine learning. If you remember, back in 2018, there was a lot of work around using deep learning for things like playing Go, and then suddenly we're playing DOTA, and we're playing Starcraft and things like that.

I guess the first revolution is – in terms of – what you can do with AI and machine learning robots. A lot of that motivated the work that we do at modl.ai because we were part of the research community, and suddenly, we looked a bit and said, "Listen, you know the technology is very new, but computers are big enough that we can start scaling this to production." So, if there ever was a time to start working on this, to make it available to the game dev community more widely, I think that was around then.

Then, over the last couple of years, we've seen generative AI come through. And I think, in many ways, that is less central to the kind of game AI we do, specifically large language models. But obviously, there's still an element of technology that plays into our link on broad behavior and what we want to do in the future and everything. The way to think about us is we're a game AI company, and a lot of what we lean on is things like imitation learning, behavioral cloning, and reinforcement learning - all those techniques that came out to scale around 2018.

80.lv: As for your AI engine, does it help you get a bunch of bots to help you not be bored when you're doing an online campaign without friends? Or maybe add more bots to your game when there are not enough concurrent users?

Christoffer Holmgård: It's much more of the former. What we do is, like, with behaviors—when I say behavior, I mean, which are the actions you take in the game? Am I shooting? Am I jumping? Where do I go? It's the gameplay that we focus on. We do that across a spectrum of complexity for the behavior. Maybe that's a way to think about it. So, we have some very, very simple parts, I would say, in a certain sense, that you can use for some purposes. And then, at times, we work with game studios to do some very advanced bots. We sit all the way on that spectrum.

The question that we sort of fundamentally ask with modl.ai is, what would you do if you had an arbitrary number of players available when you need them? And so, one thing is, of course, if you're building a multiplayer game, you're like, "Yes, I will want to make sure that versus all, there's always somebody to play against in my multiplayer game."

But also, when you're building the game, modl's AI also has a bunch of use cases. If you have players that you can put automatically into your game while you're building it, then suddenly, you start having tools that you can use for parts of the quality assurance process. You can use it to generate other kinds of data around the game. Again, you can do that without having to go and tap somebody else from your team on your shoulder and say, "Hey, can you just spend another hour playing this game to see if this works or if that works or how these dynamics play out?" That's where we say that makes sense.

80.lv: Tell us a little bit about how it works, how do bots learn how to play? Is it similar to the idea of driveatars in Forza Horizon, where you basically track how you perform, and then bots try to perform the same way?

Christoffer Holmgård: Yes, you're absolutely right. We want a total human-like experience, right? And so, what you're saying around driveatars is a pretty good example. Typically, when we work with game-playing bots, which are the ones that you'd want to put inside of your video game, we learn from the actions that players take. If you’re playing moment to moment, you're in this situation. What did you do? Then we take that data and pass it back and train the machine learning model to do the same thing. Again, what that helps you do is capture a bunch of – not random – but a lot of the human things that human players do when playing a game and get that into your dataset. Then, when you train the bot on it, you're going to get behavior typically that looks like what a human player would do in that situation.

That’s also kind of how the driveatars work. I mean, they have a lot of extra systems as well. The other approach you could take is that you can obviously train bots from scratch just by playing the game. But in that case, it's almost easier to get a superhuman bot than to get one that feels human. So, that's kind of where you need the human data to put that into the process so you can get some behavior that looks like what actual humans would do.  

Use Cases

80.lv: Can you talk about some of the use cases and companies that you worked with?

Christoffer Holmgård: We did a project with a Swedish studio called Playing Plate Games. They're an indie game developer out there, and they did a game called Forge and Fight, which is like a multiplayer fighting game.

Forge and Fight essentially is a game where you get together, you know, four persons, four players on each team, and then you play various game modes. So, there's a push-the-cart game mode where you must move a bomb as we know from other games. There's a capture-the-flag game mode, and you build your own weapons. You get cards that you earn over the course of the game, right? And when you use those cards to build your own weapons, then you must figure out how to use those weapons in this sort of action game. And so, what we did for them was we collected moment-to-moment data from what the players were doing, and that's what you're seeing here. The blue one is me, but the red one is an AI trained on player data. So, we collected what the players were doing multiple times every second, built that dataset, and changed the AI to both build its own weapons and use the weapons in combat.

And because we used player data to do so, it was actually cheap to train and run. So, it doesn't cost you 100 million dollars to train a model to do this because we already have the players showing us how to do it. And when they ended up shipping the game with this, that meant when players would join a group, if there was nobody else to play against, they would get one of these bots and still have a pretty enjoyable play experience.  

The other thing about this is that we use machine learning to get the deep behavior correct and get the weapons building right, but we still have a super system on top that the game designer can control. And that's super important when you're doing these gameplay bots because you don't want to cut the game designer out of the process, right? Because you're not going to end up with great results.

What if you're not building an action game? What if you're developing a single-player game? What are you going to be using the bots for? Depending on how familiar you are with match-three game development, the way it works is when you’re developing a match-three game, you, as the game designer, are putting together a new puzzle level for this game, right? And now, you need to understand if it's fun to play and what the statistics are. So, it's very structured, it's very methodical. You build your level, and you, as a game designer, now play it 15 times to understand how it will play out, what are the numbers, whether it is going to be fun, and all that stuff. In our case, modl.ai worked with Good Game Entertainment to figure all these out automatically.

So, what we did for Good Game Entertainment was we built a bot that would automatically play those levels for them so that the game designer doesn't have to do it. By having a bot that can play the game and gather clear data, they can make sure that the numbers that come out of it are realistic and match what the player base would do.

80.lv: It almost seems like this is sort of a substitute for potential playtests in a way, where you can playtest this stuff faster and kind of like using a synthetic test agent.

Christoffer Holmgård: It is, yeah. And that's sort of like the second use case of it. I think it's a replacement, or it's like an additive, I guess.

Normally, most studios have the game designers doing it for match-three games, for instance. So, we're interested in cutting that part out of the loop as much as they can. Because they only want to play the level maybe once and get the numbers right. That speeds up the process of making it and makes it more accurate as well because if you're using actual player data, you can use that to predict what the numbers are going to be.

The third thing we do at modl.ai is use the player bots for functional QA. And when we do that, we don't even need training. We have a basic function that you can use for functional QA and game production. Again, maybe not many people know this outside of the games industry, but 10 to 20% of your budget when you're building a game can go towards doing this kind of QA. And in spite of it, the last number I saw was about 61% of games still shipped with bugs and errors that are game-breaking. So clearly, nobody's testing enough.

So, we've made bots for QA teams where it's not even about predicting the game values. It's just about understanding if the game works or not. That's kind of the main thing for us. In that tool, we bring the engine, and you install it into your game. Once you've done that, you can set it up. We have it out for Unity and Unreal engines. For example, this is a panel in Unity that you use to integrate the bot engine. Once you have that, you just click a button, and you can use it to basically have a bot run through your game. We like to call it a Roomba or a robotic vacuum cleaner for games.

Basically, what it does for your QA team is it helps them skip over all those bits where they must check if all the walls work and if they can collect all the items and all that. But more than that, part of our AI engine is also a cloud platform that we back this with. That means that once you've set it up and you're running it, you don't even have to run the testing on your own computer. You can just upload it to us, and then you can run 100 tests at the same time. We think that’s an interesting application for virtual players as well because that allows a QA specialist to set up or configure and say, "Oh, this is how the game should be tested.” And then boom, they execute it 100 times.

The interesting thing is that both the game playing, where you play against the bots, and the win rate prediction, where you playtest the levels in the functional QA, they all come from the same thing, which is kind of like this ability to have multiple virtual players available in your game. So, those are the three use cases for modl.ai now. This last one we recently used for Salty Chronicles. They were building a narrative game, and they had the bot play through all the storylines in the game and check if it broke and if you could complete the game, and things like that. So, our expectation, and their expectation, and everybody's hope is that when the game ships – and it's going to ship on PC and multiple platforms, when it ships, it's going to be a much better experience for the player because it has been tested automatically all the time along the way. 

Getting Started

80.lv: It's great that we finally have these solutions that can be purchased off the shelf. How do people get started with your product and license it?

Christoffer Holmgård: Our whole purpose is to make this available to you if you're not EA or BioWare or whatever, right? We're going after the part of the market that wants to be able to afford this but doesn't want to have a big AI team. We're kind of an independent AI team, I guess. So, what we're doing now, is we sign people up with a bunch of hours, and after they complete those hours, they pay for how much they use it.

At the end of the day, what's happening here is that you're putting a virtual player into your game. That means we're running some computing on our side to drive the virtual player and other things around it. So, we charge you based on how much you use, and we have different licensing options available for that.

Think of us as your built-in server or other services that you use when you're building a game. We're very flexible in terms of what we offer and what we can support. Currently, we have a package for developers that gets on going at $2,500 a month, which gives them 2,000 hours of testing and game playing. But honestly, we just talk to developers, and based on their needs, we set it up to work the best for them. Our main goal is to make it flexible and affordable. 

Christoffer Holmgård, CEO and Co-Founder at modl.ai

Interview conducted by Kirill Tokarev