Efficient Solvers for Sparse Subspace Clustering

Farhad Pourkamali-Anaraki, James Folberth, Stephen Becker

Abstract: Highlights • We provide a modified version of ADMM for SSC, which has a quadratic computational cost in the size of data (instead of cubic complexity). • We present a proximal gradient method which is more efficient than ADMM-based solvers because ADMM is highly sensitive to a solver parameter. • For solving SSC with the sparsity constraint, we present a robust proximal gradient solver, which is capable of handling affine subspaces. • Experiments demonstrate the benefits of our methods in terms of computational savings and improved performance compared to popular solvers such as ADMM and OMP. Abstract Sparse subspace clustering (SSC) clusters n points that lie near a union of low-dimensional subspaces. The SSC model expresses each point as a linear or affine combination of the other points, using either ℓ1 or ℓ0 regularization. Using ℓ1 regularization results in a convex problem but requires O ( n 2 ) storage, and is typically solved by the alternating direction method of multipliers which takes O ( n 3 ) flops. The ℓ0 model is non-convex but only needs memory linear in n, and is solved via orthogonal matching pursuit and cannot handle the case of affine subspaces. This paper shows that a proximal gradient framework can solve SSC, covering both ℓ1 and ℓ0 models, and both linear and affine constraints. For both ℓ1 and ℓ0, algorithms to compute the proximity operator in the presence of affine constraints have not been presented in the SSC literature, so we derive an exact and efficient algorithm that solves the ℓ1 case with just O ( n 2 ) flops. In the ℓ0 case, our algorithm retains the low-memory overhead, and is the first algorithm to solve the SSC-ℓ0 model with affine constraints. Experiments show our algorithms do not rely on sensitive regularization parameters, and they are less sensitive to sparsity misspecification and high noise.