Sparse Local Discriminant Projections for Feature Extraction

Zhihui Lai, Zhong Jin, Jian Yang, Wai Keung Wong

Abstract: One of the major disadvantages of the linear dimensionality reduction algorithms, such as Principle Component Analysis (PCA) and Linear Discriminant Analysis (LDA), are that the projections are linear combination of all the original features or variables and all weights in the linear combination known as loadings are typically non-zero. Thus, they lack physical interpretation in many applications. In this paper, we propose a novel supervised learning method called Sparse Local Discriminant Projections (SLDP) for linear dimensionality reduction. SLDP introduces a sparse constraint into the objective function and obtains a set of sparse projective axes with directly physical interpretation. The sparse projections can be efficiently computed by the Elastic Net combining with spectral analysis. The experimental results show that SLDP give the explicit interpretation on its projections and achieves competitive performance compared with some dimensionality reduction techniques.