MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning

ICLR 2026 Conference Submission19759 Authors

Keywords: continual learning, multi-agent, overcooked, benchmark, reinforcement learning, cooperative

TL;DR: The first benchmark for continual multi-agent reinforcement learning

Abstract: Benchmarks play a crucial role in the development and analysis of reinforcement learning (RL) algorithms, with environment availability strongly impacting research. One particularly underexplored intersection is continual learning (CL) in cooperative multi-agent settings. To remedy this, we introduce **MEAL** (**M**ulti-agent **E**nvironments for **A**daptive **L**earning), the first benchmark tailored for continual multi-agent learning. Existing CL benchmarks run environments on the CPU, leading to computational bottlenecks and limiting the length of task sequences. MEAL leverages JAX for GPU acceleration, enabling continual learning across sequences of up to 100 tasks on a standard desktop PC within a few hours. Evaluating popular CL and MARL methods reveals that naïvely combining them fails to preserve network plasticity or prevent catastrophic forgetting of cooperative behaviors.

Primary Area: reinforcement learning

Submission Number: 19759