On the $O(1/T)$ Convergence of Alternating Gradient Descent–Ascent in Bilinear Games

Tianlong Nan, Shuvomoy Das Gupta, Garud Iyengar, Christian Kroer

Keywords: Two-player zero-sum games, Alternating gradient descent-ascent, Performance estimation programming

TL;DR: We show O(1/T) convergence rates of alternating gradient descent-ascent in bilinear two-player zero-sum games, with an interior Nash equilibrium or in a local region

Abstract:

We study the alternating gradient descent-ascent (AltGDA) algorithm in two-player zero-sum games.
Alternating methods, where players take turns to update their strategies, have long been recognized as simple and practical approaches for learning in games, exhibiting much better numerical performance than their simultaneous counterparts.
However, our theoretical understanding of alternating algorithms remains limited, and results are mostly restricted to the unconstrained setting.
We show that for two-player zero-sum games that admit an interior Nash equilibrium, AltGDA converges at an $O(1/T)$ ergodic convergence rate when employing a small constant stepsize. This is the first result showing that alternation improves over the simultaneous counterpart of GDA in the constrained setting.
For games without an interior equilibrium, we show an $O(1/T)$ local convergence rate with a constant stepsize that is independent of any game-specific constants.
In a more general setting, we develop a performance estimation programming (PEP) framework to jointly optimize the AltGDA stepsize along with its worst-case convergence rate.
The PEP results indicate that AltGDA may achieve an $O(1/T)$ convergence rate for a finite horizon $T$, whereas its simultaneous counterpart appears limited to an $O(1/\sqrt{T})$ rate.

Primary Area: optimization

Submission Number: 10227