Learning Multi-Agent Communication using Regularized Attention Messages

Rafael Moreira Pina, Varuna De SIlva De SIlva, Corentin Artaud

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Multi-Agent Reinforcement Learning, Communication, Attention, Message Compression

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: This paper proposes a new communication architecture to improve cooperative MARL.

Abstract: Learning how to communicate in Multi-Agent Reinforcement Learning (MARL) can be key to solve complex cooperative tasks. Recent approaches have shown the advantages of using an efficient communication architecture, tackling problems such as what, when, or whom to communicate. However, these methods still fail to solve some complex scenarios, and some of them do not evaluate the implications of having limited communication channels. In this paper, we propose Attentive Regularized Communication (ARCOMM), a new method for communication in MARL. The proposed method uses an attention module to evaluate the weight of the messages generated by the agents, together with a message regularizer that facilitates learning more meaningful messages, improving the performance of the team. We further analyse how ARCOMM reacts to situations where the messages must be compressed before being sent to other agents. Our results show that the proposed method helps, through the power of communication, to improve the performances of the agents in complex domains when compared to other methods. Furthermore, we show that, although there is a decrease of performance, agents are still capable of learning even with lossy communication. The messages learned by the agents also support the motivations for our method.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8505