Google Teaches AI to Play Quake III Arena

Recent progress in artificial intelligence through reinforcement learning (RL) has shown great success on increasingly complex single-agent environments and two-player turn-based games. However, the real-world contains multiple agents, each learning and acting independently to cooperate and compete with other agents, and environments reflecting this degree of complexity remain an open challenge. In this work, we demonstrate for the first time that an agent can achieve human-level in a popular 3D multiplayer first-person video game, Quake III Arena Capture the Flag, using only pixels and game points as input. These results were achieved by a novel two-tier optimization process in which a population of independent RL agents is trained concurrently from thousands of parallel matches with agents playing in teams together and against each other on randomly generated environments. Each agent in the population learns its own internal reward signal to complement the sparse delayed reward from winning and selects actions using a novel temporally hierarchical representation that enables the agent to reason at multiple timescales. During game-play, these agents display human-like behaviors such as navigating, following, and defending based on a rich learned representation that is shown to encode high-level game knowledge. In an extensive tournament-style evaluation, the trained agents exceeded the win-rate of strong human players both as teammates and opponents and proved far stronger than existing state-of-the-art agents. These results demonstrate a significant jump in the capabilities of artificial agents, bringing us closer to the goal of human-level intelligence.

These Arena-trained AI are said to “see” only what a player would see (the raw pixels being displayed on the screen instead of a direct feed of game data).

Google states that the resulting “FTW” (“for the win”) AI agent can compete in multiplayer CTF matches at a very high level against other bots and human opponents. The report points out that after a 40-player tournament with both human and AI players, the humans rated the AI players as “more collaborative” than their human teammates.упуп