Structured Regularization on the SPD Manifold
Keywords: Non-Convex Optimization, Riemannian Optimization, Machine Learning, Constrained Optimization
Abstract: Matrix-valued optimization tasks, including those involving symmetric positive definite (SPD) matrices, arise in a wide range of applications in machine learning, data science and statistics. Classically, such problems are solved via constrained Euclidean optimization, where the domain is viewed as a Euclidean space and the structure of the matrices (e.g., positive definiteness) enters as constraints. More recently, geometric approaches that leverage parametrizations of the problem as unconstrained tasks on the corresponding matrix manifold have been proposed. While they exhibit algorithmic benefits in many settings, they cannot directly handle additional constraints, such as side information on the solution. A remedy comes in the form of constrained Riemannian optimization methods, notably, Riemannian Frank-Wolfe and Projected Gradient Descent. However, both algorithms require potentially expensive subroutines that can introduce computational bottlenecks in practise. To mitigate these shortcomings, we propose a structured regularization framework based on symmetric gauge functions and disciplined geodesically convex programming. We show that the regularizer preserves crucial structure in the objective, including geodesic convexity. This allows for solving the regularized problem with a fast unconstrained method with a global optimality certificate. We demonstrate the effectiveness of our approach in numerical experiments on two examples, the computation of the Karcher mean of SPD matrices and Optimistic Gaussian Likelihood estimation.
Submission Number: 86
Loading