Learning To Ground Decentralized Multi-Agent Communication with Contrastive Learning

Yat Long Lo, Biswa Sengupta

Keywords: Multi-Agent Reinforcement Learning, Reinforcement Learning, Deep Reinforcement Learning, Deep Multi-Agent Reinforcement Learning, Deep Learning, Multi-Agent Systems

TL;DR: We propose a method to ground decentralized multi-agent communication based on contrastive learning to maximize the mutual information between messages within a trajectory by considering messages as incomplete views of environment states..

Abstract: For communication to happen successfully, a common language is required between agents to understand information communicated by one another. Inducing the emergence of a common language has been a difficult challenge to multi-agent learning systems. In this work, we introduce an alternative perspective to the communicative messages sent between agents, considering them as different incomplete views of the environment state. Based on this perspective, we propose a simple approach to induce the emergence of a common language by maximizing the mutual information between messages of a given trajectory in a self-supervised manner. By evaluating our method in communication-essential environments, we empirically show how our method leads to better learning performance and speed, and learns a more consistent common language than existing methods, without introducing additional learning parameters.