MARL-LNS: Efficient Multi-agent Deep Reinforcement Learning via Large Neighborhoods Search

TMLR Paper4240 Authors

Abstract: Cooperative multi-agent reinforcement learning (MARL) has emerged as a powerful paradigm for addressing complex real-world problems. However, the well-established centralized training with decentralized execution framework is hampered by the curse of dimensionality, leading to prolonged training times and inefficient convergence. In this work, we introduce MARL-LNS, a general training framework that overcomes these challenges by iteratively training on alternating subsets of agents with existing deep MARL algorithms serving as low-level trainers—without incurring any additional trainable parameters. Building on this framework, we propose three variants—Random Large Neighborhood Search (RLNS), Batch Large Neighborhood Search (BLNS), and Adaptive Large Neighborhood Search (ALNS)—each differing in its strategy for alternating agent subsets. Empirical evaluations on both the StarCraft Multi-Agent Challenge and Google Research Football environments demonstrate that our approach can reduce training time by at least 10\% while achieving comparable final performance to state-of-the-art methods.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Yaodong_Yang1

Submission Number: 4240