CoACT: Coordination via Aligned Centralized Training in Multi-Agent Reinforcement Learning

Oussama Azizi, Frans A Oliehoek, Matthijs T. J. Spaan

Keywords: Multi-Agent, Reinforcement Learning, Causal Inference, Representation learning

Abstract: Cooperative Multi-Agent Reinforcement Learning (MARL) has achieved remarkable success in complex tasks, with Centralized Training Decentralized Execution (CTDE) being the dominant paradigm for training cooperative agents. However, most CTDE methods do not account for the fact that agents are heterogeneous, each receiving observations generated by a different subset of state factors. In this work, we show that this heterogeneity can be exploited for a more efficient learning: by treating agents observation as partial views of the same underlying factors, and leveraging this structural dependency, we can can align the agents' representations during training to improve sample efficiency of existing state-of-the-art CTDE algorithms.

Paper Type: Full (minimum of 10 pages and a maximum of 16 excluding references)

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 15