LEAD: Min-Max Optimization from a Physical Perspective

Published: 15 Jun 2023, Last Modified: 02 Apr 2024Accepted by TMLREveryoneRevisionsBibTeX
Event Certifications: iclr.cc/ICLR/2024/Journal_Track
Abstract: Adversarial formulations such as generative adversarial networks (GANs) have rekindled interest in two-player min-max games. A central obstacle in the optimization of such games is the rotational dynamics that hinder their convergence. In this paper, we show that game optimization shares dynamic properties with particle systems subject to multiple forces, and one can leverage tools from physics to improve optimization dynamics. Inspired by the physical framework, we propose LEAD, an optimizer for min-max games. Next, using Lyapunov stability theory and spectral analysis, we study LEAD’s convergence properties in continuous and discrete time settings for a class of quadratic min-max games to demonstrate linear convergence to the Nash equilibrium. Finally, we empirically evaluate our method on synthetic setups and CIFAR-10 image generation to demonstrate improvements in GAN training.
Certifications: Featured Certification
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: 1. changed some part of the txt that was marked in red to black. 2. Included a citation.
Video: https://www.youtube.com/watch?v=EfwIc0GXb8E
Code: https://github.com/ReyhaneAskari/Least_action_dynamics_minmax
Supplementary Material: zip
Assigned Action Editor: ~Francisco_J._R._Ruiz1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 900