SADMA: Scalable Asynchronous Distributed Multi-agent Reinforcement Learning Training Framework

Sizhe Wang, Long Qian, Cairun Yi, Fan Wu, Qian Kou, Mingyang Li, Xingyug Chen, Xuguang Lan

Abstract: Multi-agent Reinforcement Learning (MARL) has shown significant success in solving large-scale complex decision-making problems in multi-agent systems (MAS) while facing the challenge of increasing computational cost and training time. MARL algorithms often require sufficient environment exploration to achieve good performance, especially for complex environments, where the interaction frequency and synchronous training scheme can severely limit the overall speed. Most existing RL training frameworks, which utilize distributed training for acceleration, focus on simple single-agent settings and are not scalable to extend to large-scale MARL scenarios. To address this problem, we introduce a Scalable Asynchronous Distributed Multi-Agent RL training framework called SADMA, which modularizes the training process and executes the modules in an asynchronous and distributed manner for efficient training. Our framework is powerfully scalable and provides an efficient solution for distributed training of multi-agent reinforcement learning in large-scale complex environments. Code is available at https://github.com/sadmaenv/sadma.