Eigenvector-Based Feature Extraction for Classification

Alexey Tsymbal, Seppo Puuronen, Mykola Pechenizkiy, Matthias Baumgarten, David W. Patterson

Abstract: This paper shows the importance of the use of class information in feature extraction for classification and inappropriateness of conventional PCA to feature extraction for classification. We consider two eigenvector-based approaches that take into account the class information. The first approach is parametric and optimizes the ratio of between-class variance to within-class variance of the transformed data. The second approach is a nonparametric modification of the first one based on local calculation of the between-class covariance matrix. We compare the two approaches with each other, with conventional PCA, and with plain nearest neighbor classification without feature extraction.