Stochastic Zeroth-Order Optimization under Strongly Convexity and Lipschitz Hessian: Minimax Sample Complexity
Primary Area: optimization
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: optimization, higher-order smoothness, sample complexity, optimality
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Optimization of convex functions under stochastic zeroth-order feedback has been
a major and challenging question in online learning. In this work, we consider the
problem of optimizing second-order smooth and strongly convex functions where
the algorithm is only accessible to noisy evaluations of the objective function it
queries. We provide the first tight characterization for the rate of the minimax
simple regret by developing matching upper and lower bounds. We propose an
algorithm that features a combination of a bootstrapping stage and a mirror-descent
stage. The main innovation of our approach is the usage of a gradient estimation
scheme that exploits the local geometry of the objective function, and we provide
sharp analysis for the corresponding estimation bounds.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8289
Loading