Improving Cross-Task Applicability of Parameter Sharing in Cooperative Multi-Agent Reinforcement Learning

Yurui Li, Jianyu Zhang, Li Zhang, Shijian Li, Gang Pan

Abstract: Parameter sharing is a widely adopted approach in cooperative Multi-Agent Reinforcement Learning (MARL), often achieving strong performance. However, its effectiveness can vary, as the policy’s similarity induced by parameter sharing may hinder performance in certain tasks. In this study, we propose a novel framework, termed Composite Shared Policy (CSP), to enhance the cross-task applicability of parameter sharing. CSP is designed to model multiple diverse policies concurrently, thereby introducing inherent policy diversity without relying on task-specific designs. By increasing the differences among the policies of individual agents, CSP effectively mitigates the policy similarity problem commonly associated with parameter sharing. These characteristics collectively enable CSP to improve the cross-task applicability of parameter sharing. To empirically validate the effectiveness of CSP, we implement it based on QMIX, a classic cooperative MARL method, and conduct experiments across two widely used MARL testbeds. The experimental results demonstrate that CSP significantly enhances the cross-task applicability of parameter sharing. Additionally, we conduct ablation studies to evaluate the contributions of each component within CSP. The results highlight that each component plays a critical role in the overall effectiveness of the framework. The source code is available at https://github.com/Yurui-Li/CSP.

External IDs:doi:10.3233/faia250963