Stateless Mean-Field Games: A Framework for Independent Learning with Large Populations

Batuhan Yardim, Semih Cayci, Niao He

Keywords: mean field games, independent learning, variational inequality

TL;DR: We formulate Stateless Mean-Field Games to model large scale independent learning and provide theoretical guarantees.

Abstract: Competitive games played by thousands or even millions of players are omnipresent in the real world, for instance in transportation, communications, or computer networks. However, learning in such large-scale multi-agent settings is known to be challenging due to the so-called "curse of many agents". In order to tackle large population independent learning in a general class of such problems, we formulate and analyze the Stateless Mean-Field Game (SMFG): we show that SMFG is a relevant and powerful special case of certain mean-field game formulations and a generalization of other interaction models. Furthermore, we show that SMFG can model many real-world interactions, and we prove explicit finite sample complexity guarantees with independent learning under different feedback models with repeated play. Theoretically, we contribute techniques from variational inequality (VI) literature to analyze independent learning by showing that SMFG is a VI problem at the infinite agent limit. We formulate learning and exploration algorithms which converge efficiently to approximate Nash equilibria even with finitely many agents. Finally, we validate our theoretical results in numerical examples as well as in the real-world problems of city traffic and network access.