A (dis-)information theory of revealed and unrevealed preferences

Nitay Alon, Lion Schulz, Peter Dayan, Jeffrey Rosenschein

Keywords: IPOMDP, Cognitive hierarchy, POMDP, mixed-motive game, information theory, disinformation, Signalling, Multi-agent interaction, Information Theory

TL;DR: We show how reward maximization and Theory of Mind lead to the emergence of dis-informative communication

Abstract: In complex situations involving communication, agents might attempt to mask their intentions, essentially exploiting Shannon's theory of information as a theory of misinformation. Here, we introduce and analyze a simple multiagent reinforcement learning task where a buyer sends signals to a seller via its actions, and in which both agents are endowed with a recursive theory of mind. We show that this theory of mind, coupled with pure reward-maximization, gives rise to agents that selectively distort messages and become skeptical towards one another. Using information theory to analyze these interactions, we show how savvy buyers reduce mutual information between their preferences and actions, and how suspicious sellers learn to strategically reinterpret or discard buyers' signals.

In-person Presentation: yes