Unique Sparse Decomposition of Low Rank Matrices

Dian Jin, Xin Bing, Yuqian Zhang

Abstract: The problem of finding a unique low dimensional decomposition of a given matrix has been a fundamental and recurrent problem in many areas. In this paper, we study the problem of seeking a unique decomposition of a low rank matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol Y\in \mathbb R ^{p\times n}$ </tex-math></inline-formula> that admits a sparse representation. Specifically, we consider <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol Y= \boldsymbol A \boldsymbol X $ </tex-math></inline-formula> where the matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol A\in \mathbb R^{p\times r}$ </tex-math></inline-formula> has full column rank, with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$r &lt; \min \{n,p\}$ </tex-math></inline-formula> , and the matrix <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol X\in \mathbb R^{r\times n}$ </tex-math></inline-formula> is element-wise sparse. We prove that this low rank, sparse decomposition of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol Y$ </tex-math></inline-formula> can be uniquely identified, up to some intrinsic signed permutation. Our approach relies on solving a nonconvex optimization problem constrained over the unit sphere. Our geometric analysis for its nonconvex optimization landscape shows that any <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">strict</i> local solution is close to the ground truth, and can be recovered by a simple data-driven initialization followed with any second order descent algorithm. Our theoretical findings are corroborated by numerical experiments. <xref ref-type="fn" rid="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</xref>