Accelerated Mirror Descent for Non-Euclidean Star-convex Functions

Clement LEZANE, Sophie Langer, Wouter M Koolen

Keywords: Acceleration, First order methods, Non euclidean geometry, non-convex optimization

TL;DR: Accelerated mirror descent to minimise star-convex function in non-Euclidean geometry

Abstract:

Acceleration for non-convex functions is a fundamental challenge in optimization. We revisit star-convex functions, which are
strictly unimodal on all lines through a minimizer. [1] accelerate unconstrained star-convex minimization of functions that are smooth with respect to the Euclidean norm. To do so, they add a certain binary search step to gradient descent.

In this paper, we accelerate unconstrained star-convex minimization of functions that are weakly smooth with respect to an arbitrary norm. We add a binary search step to mirror descent, generalize the approach and refine its complexity analysis.

We prove that our algorithms have sharp convergence rates for star-convex functions with $\alpha$-Holder continuous gradients and demonstrate that our rates are nearly optimal for $p$-norms.

[1] Oliver Hinder, Aaron Sidford, and Nimit Sohoni. Near-optimal methods for minimizing star-convex
functions and beyond

Submission Number: 84