RACCOON: Regret-based Adaptive Curricula for Cooperation
Keywords: unsupervised environment design, multi-agent reinforcement learning, cooperation, autocurricula, ad-hoc teamwork, zero-shot coordination
TL;DR: A novel method using regret-based prioritised sampling of diverse training partners and tasks to improve ad-hoc teamwork.
Abstract: Overfitting to training partners is a common problem in fully-cooperative multi-agent settings, leading to poor zero-shot transfer to novel partners. A popular solution is to train an agent with a diverse population of training partners. However, previous work lacks a principled approach for selecting partners from this population during training, usually sampling at random. We argue that partner sampling is an important and overlooked problem, and motivated by the success of regret-based Unsupervised Environment Design, we propose Regret-based Adaptive Curricula for Cooperation (RACCOON), a novel a method which prioritises high-regret partners and tasks. We test RACCOON in the Overcooked environment, and demonstrate that it leads to sample efficiency gains and increased robustness across diverse partners and tasks, compared with strong baselines. We further analyse the nature of the induced curricula, and conclude with discussions on the limitations of cooperative regret and directions for future work.
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10348
Loading