Non-convex matrix sensing: Breaking the quadratic rank barrier in the sample complexity

Dominik Stöger, Yizhe Zhu

Keywords: non-convex optimization, factorized gradient descent, matrix sensing, sample complexity, virtual sequences

TL;DR: For symmetric matrix sensing, this paper shows that gradient descent recovers the ground truth with sample complexity which corresponds to the number of degrees of freedom of the ground truth

Abstract:

For the problem of reconstructing a low-rank matrix from a few linear measurements,
two classes of algorithms have been widely studied in the literature:
convex approaches based on nuclear norm minimization,
and non-convex approaches that use factorized gradient descent.
Under certain statistical model assumptions,
it is known that nuclear norm minimization recovers the ground truth
as soon as the number of samples scales linearly with the number of degrees of freedom of the ground-truth.
In contrast, while non-convex approaches are computationally less expensive,
existing recovery guarantees assume
that the number of samples scales at least quadratically with the rank $r$ of the ground-truth matrix.
In this paper, we close this gap
by showing that
the non-convex approaches can be as efficient as nuclear norm minimization in terms of sample complexity.
Namely, we consider the problem of
reconstructing a positive semidefinite matrix from a few Gaussian measurements.
We show that factorized gradient descent with spectral initialization
converges to the ground truth with a linear rate
as soon as the number of samples
scales with $ \Omega (rd\kappa^2)$, where $d$ is the dimension,
and $\kappa$ is the condition number of the ground truth matrix.
This improves the previous rank-dependence in the sample complexity of non-convex matrix factorization from quadratic to linear.
Our proof relies on a probabilistic decoupling argument,
where we show that the gradient descent iterates are only
weakly dependent on the individual entries of the measurement matrices.
We expect that our proof technique is of independent interest for other non-convex problems.

Submission Number: 21