Accelerated Riemannian Optimization: Handling Constraints to Bound Geometric Penalties

David Martínez-Rubio, Sebastian Pokutta

Keywords: Riemannian optimization, geodesic convexity, acceleration, proximal methods

TL;DR: We propose accelerated first-order methods for Riemannian optimization in Hadamard manifolds by using a proximal method that we design. We can work without undesirable assumptions previous accelerated works made

Abstract:

We propose a globally-accelerated, first-order method for the optimization of smooth and (strongly or not) geodesically-convex functions in Hadamard manifolds. Our algorithm enjoys the same convergence rates as Nesterov's accelerated gradient descent, up to a multiplicative geometric penalty and log factors.
Crucially, we can enforce our method to stay within a compact set we define. Prior fully accelerated works resort to assuming that the iterates of their algorithms stay in some pre-specified compact set, except for two previous methods, whose applicability is limited to local optimization and to spaces of constant curvature, respectively. Achieving global and general Riemannian acceleration without iterates assumptively staying in the feasible set was asked as an open question in (Kim & Yang, 2022), which we solve for Hadamard manifolds.
In our solution, we show that we can use a linearly convergent algorithm for constrained strongly g-convex smooth problems to implement a Riemannian inexact proximal point operator that we use as a subroutine, which is of independent interest.