A Framework for Finding Local Saddle Points in Two-Player Zero-Sum Black-Box Games

Shubhankar Agarwal, Hamzah I Khan, Sandeep P. Chinchali, David Fridovich-Keil

Abstract: Saddle point optimization is a critical problem employed in numerous real-world applications, including portfolio optimization, generative adversarial networks, and robotics. It has been extensively studied in cases where the objective function is known and differentiable. Existing work in black-box settings with unknown objectives that can only be sampled either assumes convexity-concavity in the objective to simplify the problem or operates with noisy gradient estimators. In contrast, we introduce a framework inspired by Bayesian optimization which utilizes Gaussian processes to model the unknown (potentially nonconvex-nonconcave) objective and requires only zeroth-order samples. Our approach frames the saddle point optimization problem as a two-level process which can flexibly leverage existing general-sum Nash game solvers to solve for saddle points of zero-sum games. The upper level of our framework produces a model of the objective function by sampling in promising locations, and the lower level of our framework uses the existing model to frame and solve a general-sum game to identify locations to sample. This lower level procedure can be designed in complementary ways, and we demonstrate the flexibility of our approach by introducing variants which appropriately trade off between factors like runtime, the cost of function evaluations, and the number of available initial samples. We experimentally demonstrate these algorithms on synthetic and realistic datasets in black-box nonconvex-nonconcave settings, showcasing their ability to efficiently locate local saddle points in these contexts.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=pTKRIF2MRC&noteId=pTKRIF2MRC

Changes Since Last Submission: Fixed the font error that caused the desk rejection, and adjusted text/figures to fit 12 pages of main content.

Assigned Action Editor: ~Kamyar_Azizzadenesheli1

Submission Number: 4379