Semi-Anchored Gradient Methods for Nonconvex-Nonconcave Minimax Problems

Sucheol Lee, Donghwan Kim

Primary Area: optimization

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Optimization, Minimax, PDHG, nonconvex-nonconcave, Weak-MVI

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose a new nonlinear variant of PDHG, named semi-anchoring, which converges for structured nonconvex-nonconcave minimax problems.

Abstract:

Nonconvex-nonconcave minimax problems are difficult to optimize by gradient methods. The extragradient method, proven to outperform the gradient descent ascent, has become standard but there is still room for improvement. On the other hand, under a bilinear setting, the primal-dual hybrid gradient (PDHG) method is one of the most popular methods. This was studied on a general convex-concave problem, but it has not been found useful in a more general nonconvex-nonconcave minimax problem. In this paper, we demonstrate its natural extension to a structured nonconvex-nonconcave minimax problem, whose saddle-subdifferential operator satisfies the weak Minty variational inequality condition, showing its potential. This new nonlinear variant of PDHG, named semi-anchored (SA) gradient method,
is built upon the theory of Bregman proximal point method. This consequently provides a worst-case convergence rate, in terms of a new optimality measure for nonconvex-nonconcave minimax optimization, making it interesting on its own. We further illustrate the potential of the semi-anchoring by providing a numerical experiment on fair classification problem, in comparison with the extragradient.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9088