SORA: Improving Multi-agent Cooperation with a Soft Role Assignment Mechanism

Guangchong Zhou, Zhiwei Xu, Zeren Zhang, Guoliang Fan

Abstract: Role-based multi-agent reinforcement learning (MARL) holds the promise of achieving scalable multi-agent cooperation by decomposing complex tasks through the concept of roles and has enjoyed great success in various tasks. However, conventional role-based MARL methods typically assign a single role to each agent, limiting the agent’s behavior in certain scenarios. In real life, an individual usually performs multiple responsibilities in a given task. To meet such situations, we propose a novel soft role assignment (SORA) process that enables an agent to play multiple roles simultaneously. Concretely, SORA first generates a role distribution via the attention mechanism to interpret the agent’s identity as a combination of different roles. To ensure consistent behavior with an agent’s assigned role, we also introduce role-specific Q networks for decision-making. By virtue of these advances, our proposed method makes a prominent improvement over the prior state-of-the-art approaches on StarCraft multi-agent challenges and Google Research Football.