Generalized Sparse Additive Model with Unknown Link Function

Peipei Yuan, Xinge You, Hong Chen, Xuelin Zhang, Qinmu Peng

Abstract: Generalized additive models (GAMs) have been successfully applied to high dimensional data. However, most existing methods cannot capture the high level feature patterns from complex data. To alleviate this problem, we propose a new sparse additive model, named generalized sparse additive model with unknown link function (GSAMUL), in which the component functions are estimated by B-spline basis and the unknown link function is estimated by a multi-layer perceptron (MLP) network. Furthermore, $\mathscr{l}_{2.1}$ -norm regularizer is used for variable selection. The proposed GSAMUL can realize both variable selection and hidden interaction. We integrate this estimation into a bilevel optimization problem, where the data is split into training set and validation set. In theory, we provide the guarantees about the convergence of the approximate procedure. In applications, experimental evaluations on both synthetic and real world data sets consistently validate the effectiveness of GSAMUL.