Modelling the Dynamics of Regret Minimization in Large Agent Populations: a Master Equation Approach

Zhen Wang, Chunjiang Mu, Shuyue Hu, Chen Chu, Xuelong Li

Abstract: Understanding the learning dynamics in multiagent systems is an important and challenging task. Past research on multi-agent learning mostly focuses on two-agent settings. In this paper, we consider the scenario in which a population of infinitely many agents apply regret minimization in repeated symmetric games. We propose a new formal model based on the master equation approach in statistical physics to describe the evolutionary dynamics in the agent population. Our model takes the form of a partial differential equation, which describes how the probability distribution of regret evolves over time. Through experiments, we show that our theoretical results are consistent with the agent-based simulation results.