Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learning

Sheng Li, Yutai Zhou, Ross E. Allen, Mykel J. Kochenderfer

Abstract: Communication is an important factor that en-ables agents to work cooperatively in multi-agent reinforcement learning (MARL) contexts. Prior work used continuous message communication whose high representational capacity comes at the expense of interpretability. Allowing agents to learn their own discrete emergent message communication protocols can increase the interpretability for human designers and other agents. This paper proposes a method to generate discrete messages analogous to human languages. Discrete message communication is achieved by a broadcast-and-listen mecha-nism based on self-attention. We show that discrete message communication has performance comparable to continuous message communication but with a much smaller vocabulary size. Discrete message communication protocols can potentially be used for human-agent interaction.