Last Iterate Convergence in Monotone Mean Field Games

Noboru Isobe, Kenshi Abe, Kaito Ariu

Keywords: mean-field game, learning in games

TL;DR: We prove convergence of an efficient method to solve unregularized MFGs under non-strict monotonicity.

Abstract:

In the Lasry--Lions framework, Mean-Field Games (MFGs) model interactions among an infinite number of agents.
However, existing algorithms either require strict monotonicity or only guarantee the convergence of averaged iterates, as in Fictitious Play in continuous time.
We address this gap with the following theoretical result.
First, we prove that the last-iterated policy of a proximal-point (PP) update with KL regularization converges to an MFG equilibrium under non-strict monotonicity.
Second, we see that each PP update is equivalent to finding the equilibria of a KL-regularized MFG.
We then prove that this equilibrium can be found using mirror descent with an exponential last-iterate convergence rate.
Building on these insights, we propose the Approximate Proximal-Point ($\mathtt{APP}$) algorithm, which approximately implements the PP update via a small number of Mirror Descent steps.
Numerical experiments on standard benchmarks confirm that the $\mathtt{APP}$ algorithm reliably converges to the unregularized mean-field equilibrium without time-averaging.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 5548