Why Emergent Communication is Repulsive
Abstract: With the success of deep reinforcement learning, there has been a resurgence of interest in situated emergent communication research. Properties of successful emergent communication have been identified which typically involve auxiliary losses that ensure a trade-off between ensuring diversity of message-action pairs, conditioned on observations, and consistency, when the reward acquired is significant. In this work, we draw theoretically connections between these auxiliary losses and the probabilistic framework of repulsive point processes. We show how in fact these auxiliary losses are promoting repulsive point processes, as well as outline ways in which the practitioner could utilise these repulsive point processes directly. We hope this newfound connection between language and repulsive point processes offers new avenues of research for the situated language researcher or probabilistic modeller.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~DJ_Strouse1
Submission Number: 22
Loading