LogIn
Google DeepMind presented SIMA, an agent that can follow natural-language instructions to complete in-game tasks.
Image credit: Google DeepMind
We've seen AI agents play games before (Minecraft is especially popular) but why would we want to merely watch them? We want to play together! And we might be able to with SIMA, Scalable Instructable Multiworld Agent, which can perform some in-game actions so humans can have more fun playing with it.
Created by Google DeepMind, SIMA can follow natural-language instructions to perform tasks like moving, opening the map, and interacting with objects.
"To expose SIMA to many environments, we’ve built a number of partnerships with game developers for our research. We collaborated with eight game studios to train and test SIMA on nine different video games, such as No Man’s Sky by Hello Games and Teardown by Tuxedo Labs. Each game in SIMA’s portfolio opens up a new interactive world, including a range of skills to learn, from simple navigation and menu use, to mining resources, flying a spaceship, or crafting a helmet."
Image credit: Google DeepMind
This "generalist AI agent for 3D virtual settings" connects language with in-game behavior and "comprises a model designed for precise image-language mapping and a video model that predicts what will happen next on-screen." It doesn’t need the game's source code or APIs, it wants the images on the screen and natural-language instructions. The creators think SIMA can potentially interact with any virtual environment.
"We want our future agents to tackle tasks that require high-level strategic planning and multiple sub-tasks to complete, such as “Find resources and build a camp”. This is an important goal for AI in general, because while Large Language Models have given rise to powerful systems that can capture knowledge about the world and generate plans, they currently lack the ability to take actions on our behalf."
SIMA managed to complete nearly 1500 unique in-game tasks and performed better than AI agents playing only one game. In the future, the researchers hope to develop a new wave of generalist, language-driven AI agents that can do more complex tasks and become helpful to people online and in the real world.
Learn more about it here and join our 80 Level Talent platform and our Telegram channel, follow us on Instagram, Twitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.
Keep reading
You may find these articles interesting