Joint Feature Selection and Extraction With Sparse Unsupervised Projection

Jingyu Wang, Lin Wang, Feiping Nie, Xuelong Li

Abstract: Feature selection and feature extraction, in the field of data dimensionality reduction, are the two main strategies. Nevertheless, each of these two strategies has its own advantages and disadvantages. The features chosen by feature selection method have complete physical meaning. However, feature selection cannot reveal the implicit structural information of the samples. In this article, the methods proposed by us combine both feature selection and feature extraction, called joint feature selection and extraction with sparse unsupervised projection (SUP) and graph optimization SUP (GOSUP). A constraint on the number of nonzero rows of the projection matrix is added, which ensures the sparsity of the projection matrix, and only the features corresponding to the nonzero rows of the projection matrix are selected for the feature extraction procedure. We invoke a newly proposed algorithm to tackle this constrained optimization problem. A new concept of “purification matrix” is invented, the use of which could better eliminate meaningless information of samples in subspace. The performance on several datasets verifies the effectiveness of the proposed method for data dimensionality reduction.