Multi-Agent Reinforcement Learning for Heterogeneous Large-Scale Blotto Games

Yuanyuan Zhang, Gang Xiao, Feng Ye, Lingtao Xue, Tong Li, Zhipeng Du

Keywords: Multi-Agent Reinforcement Learning;Blotto Game;Large-Scale Resource Allocation;Value Function Factorization

TL;DR: An multi-agent reinforcement learning framework that solves large-scale Blotto games via parameter sharing and hierarchical observation design.

Abstract: The Colonel Blotto game, a classical resource allocation model in game theory, presents significant computational challenges when extended to large-scale heterogeneous settings due to combinatorial strategy space explosion and agent heterogeneity. We introduce a multi-agent reinforcement learning framework that effectively solves ultra-large-scale heterogeneous Blotto games involving thousands of agents and dozens of battlefields. Our approach formulates the problem as a decentralized partially observable Markov decision process and proposes a dual-path algorithmic architecture: Group-Mix enables precise credit assignment through type-aware value decomposition, while H-PPO ensures training stability via hierarchical curriculum learning. Theoretical analysis establishes the viability of centralized training with decentralized execution for Blotto games and demonstrates the strategy space compression achieved through type-sharing mechanisms. Experimental results validate that our method maintains stable learning and generates effective strategies in complex scenarios with 1,000 agents and 20 battlefields, demonstrating practical efficacy in ultra-large-scale settings previously considered computationally intractable.

Primary Area: reinforcement learning

Submission Number: 15401