Adapting Regularized Low-Rank Models for Parallel Architectures

Derek Driggs, Stephen Becker, Aleksandr Y. Aravkin

Abstract: We introduce a reformulation of regularized low-rank recovery models to take advantage of GPU, multiple CPU, and hybridized architectures. Low-rank recovery often involves nuclear norm minimization through iterative thresholding of singular values. These models are slow to fit and difficult to parallelize because of their dependence on computing a singular value decomposition at each iteration. Regularized low-rank recovery models also incorporate nonsmooth terms to separate structured components (e.g., sparse outliers) from the low-rank component, making these problems more difficult. Using Burer--Monteiro splitting and marginalization, we develop a smooth, nonconvex formulation of regularized low-rank recovery models that can be fit with first-order solvers. Replacing convex problems with nonconvex programs that can be solved more quickly is a well-studied approach to low-rank recovery; here we show that handling the regularization terms carefully allows for greater speedup. Using robust principal component analysis (RPCA) as an example, we compare our approach to other convex, nonconvex, and communication-avoiding algorithms on the GPU, and show that ours is an order-of-magnitude faster. We also show that this acceleration allows for new applications of RPCA, including real-time background subtraction and magnetic resonance imaging analysis.