Generalized Correlation Shifting for Lasso

Izuru Miyazaki, Hironori Fujisawa

TL;DR: An adaptive preconditioner that induces low-rank structure in the design matrix improves Lasso’s sign recovery in high-dimensional settings.

Abstract: The Lasso has been widely used in a high-dimensional setting, but its estimation accuracy may become inadequate when the covariates are highly correlated or when the number of covariates is extremely large. To overcome this problem, we propose a novel preconditioner that adaptively induces a low-rank structure in the design matrix. The proposed preconditioner achieves a higher probability of sign correctness under some conditions. We establish theoretical guarantees showing that our method dominates the standard Lasso, and we further demonstrate its superiority over the correlation shifting. To validate its practical effectiveness, we conducted numerical experiments on synthetic and semi-real datasets, and the proposed method presented better performance than existing methods.

Submission Number: 1092