Guarantees of a Preconditioned Subgradient Algorithm for Overparameterized Asymmetric Low-rank Matrix Recovery

Paris Giampouras, HanQin Cai, Rene Vidal

TL;DR: Linear convergence rates of an overparameterized preconditioned subgradient algorithm for robust low-rank matrix estimation.

Abstract: In this paper, we focus on a matrix factorization-based approach for robust recovery of low-rank asymmetric matrices from corrupted measurements. We propose an Overparameterized Preconditioned Subgradient Algorithm (OPSA) and provide, for the first time in the literature, linear convergence rates independent of the rank of the sought asymmetric matrix in the presence of gross corruptions. Our work goes beyond existing results in preconditioned-type approaches addressing their current limitation, i.e., the lack of convergence guarantees in the case of asymmetric matrices of unknown rank. By applying our approach to (robust) matrix sensing, we highlight its merits when the measurement operator satisfies a mixed-norm restricted isometry property. Lastly, we present extensive numerical experiments that validate our theoretical results and demonstrate the effectiveness of our approach for different levels of overparameterization and corruption from outliers.

Lay Summary:

This paper focuses on a problem that appears in many areas like image processing, data science, and machine learning: how to recover a clean and complete version of a matrix when some of the data is corrupted. To tackle this, we propose a new method called OPSA. OPSA is designed to work even when the structure of the original matrix is not fully known and when the data is badly corrupted. What's special about our method is that:

- It can handle cases where the matrix isn't nicely balanced (asymmetric),
- It doesn’t require knowing in advance how complex the original matrix is (its “rank”),
- It still works well even if the corruption in the data is pretty severe, no matter how complicated the matrix is.

We have tested our method in various challenging scenarios, and it consistently performed well. We also backed up our claims with mathematical proofs to show that our approach is not only practical but also theoretically sound.

Link To Code: https://github.com/caesarcai/OPSA

Primary Area: Optimization->Non-Convex

Keywords: Matrix factorisation, low-rank, matrix sensing, preconditioned subgradient

Submission Number: 11771