A semi-parametric approach to feature selection in high-dimensional linear regression models

yuyang liu, Pengfei Pi, Shan Luo

Abstract: We propose a novel semi-parametric approach to feature selection in high-dimensional linear regression models. This sequential procedure is robust to the unknown error distribution including heavy-tailed distributions. At each step of this procedure, we add the feature with the largest absolute value of the estimated partial profile score into the model. The procedure terminates when a model selection criterion is met. Theoretically, we establish this procedure’s selection consistency under regular conditions. Computationally, extensive numerical studies together with a real data application are provided to demonstrate its advantage over existing representative methods in terms of selection accuracy and computation cost.