everyone
since 13 Oct 2023">EveryoneRevisionsBibTeX
In minimax optimization, the extragradient (EG) method has been extensively studied because it outperforms the gradient descent-ascent (GDA) method in both strongly-convex-strongly-concave (SC-SC) and convex-concave (C-C) problems. However, stochastic EG (SEG) has seen limited success, as it is known to converge only up to neighborhoods of equilibria for C-C problems. Motivated by the recent progress in analysis of shuffling-based stochastic optimization methods, we investigate the convergence of shuffling-based SEG in finite-sum minimax problems, in search of improved convergence guarantees for SEG under minimal algorithm modifications. Our analysis reveals that both random reshuffling and the recently proposed flip-flop shuffling (Rajput et al., 2022) alone cannot fix the nonconvergence issue in C-C problems. However, with an additional simple trick called anchoring, we develop the SEG with flip-flop anchoring (SEG-FFA) method which successfully converges in C-C problems. We also show upper and lower bounds in the SC-SC setting, demonstrating that SEG-FFA has a provably faster convergence rate compared to other shuffling-based methods as well.