Sparsity via Sparse Group k-max Regularization

Qinghua Tao, Xiangming Xi, Jun Xu, Johan A. K. Suykens

Abstract: For the linear inverse problem with sparsity constraints, the $l_{0}$ regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost optimal solutions or approximate the $l_{0}$ regularization with its convex counterparts. In this paper, we propose a novel regularization technique, namely the sparse group $k$ -max regularization. This approach has the advantages of enhancing both the group-wise and in-group sparsity without imposing any additional constraints on the magnitude of variables in each group. Our approach approximates the $l_{0}$ norm more closely than other relaxation methods, making it especially important for problems with variables at varying scales. We have also established an iterative soft thresholding algorithm and have provided local optimality conditions and complexity analysis. We have tested our method with numerical experiments on both synthetic and real-world datasets, and our results show that our technique is effective and flexible.