On the Verification Complexity of Deterministic Nonsmooth Nonconvex Optimization
Primary Area: optimization
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: nonconvex nonsmooth optimization, optimization theory, verification complexity
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: We study the complexity of deterministic verifiers for nonsmooth nonconvex optimization when interacting with an omnipotent prover and we obtain the first exponential lower bounds for the problem.
In the nonsmooth setting, Goldstein stationary points constitute the solution concept recent works have focused on. Lin, Zheng and Jordan (NeurIPS '22) show that
even uniform Goldstein stationary points of a nonsmooth nonconvex function can be found efficiently via randomized zeroth order algorithms, under a Lipschitz condition.
As a first step,
we show that verification of Goldstein stationarity via determistic algorithms is possible
under access to exact queries and first order oracles. This is done via a natural but novel connection with Carathéodory's theorem.
We next show that even verifying uniform Goldstein points
is intractable for deterministic zeroth order algorithms. Therefore, randomization is necessary (and sufficient) for efficiently finding uniform Goldstein stationary points via zeroth order algorithms.
Moreover, for general (nonuniform) Goldstein stationary points, we prove that any deterministic zeroth order verifier that is restricted to queries in a lattice needs a number of queries that is exponential in the dimension.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2648
Loading