Sparse Canonical Correlation Analysis via Smooth Non-Convex $\ell_{0}$ Surrogates and Iterative Minorization–Maximization

ICLR 2026 Conference Submission16257 Authors

Keywords: canonical correlation analysis (CCA), $\ell_0$ cardinality constraint, smooth surrogate penalties

Abstract: Canonical correlation analysis (CCA) is a core tool to uncover linear associations between two datasets. In high-dimensional settings, however, it is prone to overfitting and lacks interpretability. Enforcing exact sparsity via $\ell_0$ constraints can improve interpretability but leads to an intractable combinatorial problem. We propose a novel framework for sparse CCA that replaces the $\ell_0$ cardinality constraint with tight smooth concave surrogates (power, logarithmic, and exponential forms), preserving support control without ad hoc thresholds. We solve the resulting nonconvex program via a minorization–maximization algorithm, yielding a generalized eigenvalue subproblem at each step. We prove that as the smoothing parameter vanishes, the surrogate formulation converges to the exact $\ell_0$ solution with explicit suboptimality bounds. We further reformulate the objective as a rank-constrained semidefinite program and use randomized Gaussian rounding to extract sparse canonical directions. Empirical results on six benchmark datasets demonstrate that our method enforces exact sparsity levels, delivers superior canonical correlations and support recovery, and offers markedly improved scalability compared to state‐of‐the‐art SCCA algorithms.

Primary Area: optimization

Submission Number: 16257