Multi-agent Learning Dynamics: A Survey

H. Jaap van den Herik, Daniel Hennes, Michael Kaisers, Karl Tuyls, Katja Verbeeck

Abstract: In this paper we compare state-of-the-art multi-agent reinforcement learning algorithms in a wide variety of games. We consider two types of algorithms: value iteration and policy iteration. Four characteristics are studied: initial conditions, parameter settings, convergence speed, and local versus global convergence. Global convergence is still difficult to achieve in practice, despite existing theoretical guarantees. Multiple visualizations are included to provide a comprehensive insight into the learning dynamics.