The Cell Must Go On: Agar.io for Continual Reinforcement Learning

ICLR 2026 Conference Submission20274 Authors

Keywords: Reinforcement Learning, Continual Reinforcement Learning, Games

TL;DR: This paper presents AgarCL, a continual RL benchmark based on Agar.io, designed to study learning in dynamics, non-episodic environments

Abstract: Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fixed for evaluation. Such an approach is well suited to environments the agent perceives as changing, which renders any static policy ineffective over time. The few simulators explicitly designed for empirical research in continual RL are often limited in scope or complexity, and it is now common for researchers to modify episodic RL environments by artificially incorporating abrupt task changes during interaction. In this paper, we introduce AgarCL, a research platform for continual RL that allows for a progression of increasingly sophisticated behaviour. AgarCL is based on the game Agar.io, a non-episodic, high-dimensional problem featuring stochastic, ever-evolving dynamics, continuous actions, and partial observability. Additionally, we provide benchmark results reporting the performance of DQN, PPO, and SAC in both the primary, challenging continual RL problem, and across a suite of smaller tasks within AgarCL, each of which isolates aspects of the full environment and allow us to characterize the challenges posed by the different aspects of the game.

Primary Area: datasets and benchmarks

Submission Number: 20274