MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning

Tristan Tomilin, Luka van den Boogaard, Samuel Garcin, Bram Grooten, Yali Du, Meng Fang, Mykola Pechenizkiy

Keywords: continual learning, multi-agent, overcooked, benchmark, reinforcement learning, cooperation

TL;DR: The first benchmark for continual multi-agent reinforcement learning

Abstract: Benchmarks play a crucial role in the development and analysis of reinforcement learning (RL) algorithms, with environment availability strongly impacting research. One particularly underexplored intersection is continual learning (CL) in cooperative multi-agent settings. To remedy this, we introduce **MEAL** (**M**ulti-agent **E**nvironments for **A**daptive **L**earning), the first benchmark tailored for continual multi-agent learning. Existing CL benchmarks run environments on the CPU, leading to computational bottlenecks and limiting the length of task sequences. MEAL leverages JAX for GPU acceleration, enabling continual learning across sequences of up to 100 tasks on a standard desktop PC within a few hours. Evaluating popular CL and MARL methods reveals that naïvely combining them fails to preserve network plasticity or prevent catastrophic forgetting of cooperative behaviors.

Code URL: https://anonymous.4open.science/r/anonrepo-6013/README.md

Primary Area: Data for Reinforcement learning (e.g., decision and control, planning, hierarchical RL, robotics)

Submission Number: 2022