Robust and Faster Zeroth-Order Minimax Optimization: Complexity and Applications

Weixin An, Yuanyuan Liu, Fanhua Shang, Hongying Liu

Keywords: zeroth-order algorithms, minimax problems, robustness, complexity, regression

TL;DR: We propose a robust and faster zeroth-order algorithm to solve nonconvex minimax problems.

Abstract: Many zeroth-order (ZO) optimization algorithms have been developed to solve nonconvex minimax problems in machine learning and computer vision areas. However, existing ZO minimax algorithms have high complexity and rely on some strict restrictive conditions for ZO estimations. To address these issues, we design a new unified ZO gradient descent extragradient ascent (ZO-GDEGA) algorithm, which reduces the overall complexity to $\mathcal{O}(d\epsilon^{-6})$ to find an $\epsilon$-stationary point of the function $\psi$ for nonconvex-concave (NC-C) problems, where $d$ is the variable dimension. To the best of our knowledge, ZO-GDEGA is the first ZO algorithm with complexity guarantees to solve stochastic NC-C problems. Moreover, ZO-GDEGA requires weaker conditions on the ZO estimations and achieves more robust theoretical results. As a by-product, ZO-GDEGA has advantages on the condition number for the NC-strongly concave case. Experimentally, ZO-GDEGA can generate more effective poisoning attack data with an average accuracy reduction of 5\%. The improved AUC performance also verifies the robustness of gradient estimations.

Primary Area: Optimization (convex and non-convex, discrete, stochastic, robust)

Submission Number: 8287