Superhuman AI for Generals.io Using Self-Play Reinforcement Learning

Matej Straka, Martin Schmid

Keywords: self-play, real-time strategy, superhuman-ai, magnetic mirror descent, jax environment

TL;DR: superhuman ai for online game generals.io using magnetic mirror descent together with jax-based game simulator for various game modes

Abstract: We present a superhuman AI agent for Generals.io, a real-time strategy game that requires both long-horizon planning and short-term tactics under strong imperfect information. Trained on a modest compute budget, our agent reaches #1 on the public 1v1 leaderboard of over 5,000 players. We reconsider the algorithmic machinery used in previous approaches and show that simple, general-purpose methods suffice: we drop behavior cloning, potential-based reward shaping, and population-based self-play in favor of a generic policy-gradient training loop. Alongside the agent we release a JAX-native simulator supporting 1v1, 2v2, and arbitrary-N free-for-all under a shared interface, enabling research on cooperative and general-sum multi-agent play.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Paper Type: Standard paper

Submission Number: 6