Nonconvex Sparse Logistic Regression With Weakly Convex Regularization

Xinyue Shen, Yuantao Gu

Abstract: In this paper, we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem. The idea is based on the finding that a weakly convex function as an approximation of the ℓ0 pseudo norm is able to better induce sparsity than the commonly used ℓ1 norm. For a class of weakly convex sparsity inducing functions, we prove the nonconvexity of the corresponding problem and study its local optimality conditions and the choice of the regularization parameter. Despite the nonconvexity, a method based on proximal gradient descent is used to solve the general weakly convex sparse logistic regression, and its convergence behavior is studied theoretically. Then, the general framework is applied to a specific weakly convex function, and a local optimality condition and a bound on the logistic loss at a local optimum are provided. The solution method is instantiated in this case as an iterative firm-shrinkage algorithm, and a Nesterov acceleration is used with a convergence guarantee. Its effectiveness is demonstrated in numerical experiments by both randomly generated and real datasets.

External IDs:doi:10.1109/tsp.2018.2824289