Learning non-linear classifiers with a sparsity constraint using L1 regularization

Mathieu Blondel, Kazuhiro Seki, Kuniaki Uehara

Abstract: When combined with kernels, Support Vector Machines (SVMs) often achieve outstanding accuracy. This comes, however, at the cost of very expensive predictions. To address this issue, we consider the problem of learning a non-linear classifier with a sparsity constraint. Using ℓ1-regularization, we show that this reduces to a univariate parameter search problem, which we show how to solve efficiently. Experiments show that our approach, while leading to much sparser models, is competitive with unconstrained kernel SVMs, both in terms of accuracy and training time.