Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles

Abstract: This paper explores the performance of a random Gaussian smoothing zeroth-order (ZO) scheme for minimising quasar-convex (QC) and strongly quasar-convex (SQC) functions in both unconstrained and constrained settings. For the unconstrained problem, we establish the ZO algorithm's convergence to a global minimum along with its complexity when applied to both QC and SQC functions. For the constrained problem, we introduce the new notion of proximal-quasar-convexity and prove analogous results to the unconstrained case. Specifically, we derive complexity bounds and prove convergence of the algorithm to a neighbourhood of a global minimum whose size can be controlled under a variance reduction scheme. Beyond the theoretical guarantees, we demonstrate the practical implications of our results on several machine learning problems where quasar-convexity naturally arises, including linear dynamical system identification and generalised linear models.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: We have submitted a revised manuscript after addressing all the issues raised by the reviewers. The parts of the revised manuscript that underwent significant changes as compared to our original submission are highlighted in blue. The updated version of the supplementary materials has been uploaded.

Assigned Action Editor: ~Meisam_Razaviyayn1

Submission Number: 7250