The Solution Path of the Group Lasso

Aaron Mishkin, Mert Pilanci

Keywords: group lasso, solution function, regularization path, sparsity, group sparsity, convex optimization

TL;DR: We prove continuity of the regularization path for the linear group lasso problem.

Abstract: We prove continuity of the solution path for the group lasso, a popular method of computing group-sparse models. Unlike the more classical lasso method, the group lasso solution path is non-linear and cannot be evaluated algorithmically. To circumvent this, we first characterize the group lasso solution set and then show how to construct an implicit function for the min-norm path. We prove our implicit representation is continuous almost everywhere and extend this to continuity everywhere when the group lasso solution is unique. These results can be viewed as extending solution path analyses from the lasso to the group lasso and imply that grid-search is a sensible approach to hyper-parameter selection. Our work applies to linear models as well as convex reformulations of neural networks and provides new tools for understanding solution paths of shallow ReLU models.