Supervised Principal Component Analysis Via Manifold Optimization

Alexander Ritchie, Clayton D. Scott, Laura Balzano, Daniel Kessler, Chandra Sekhar Sripada

Abstract: High dimensional prediction problems are pervasive in the scientific community. In practice, dimensionality reduction (DR) is often performed as an initial step to improve prediction accuracy and interpretability. Principal component analysis (PCA) has been utilized extensively for DR, but does not take advantage of outcome variables inherent in the prediction task. Existing approaches for supervised PCA (SPCA) either take a multi-stage approach or incorporate supervision indirectly. We present a manifold optimization approach to SPCA that simultaneously solves the prediction and dimensionality reduction problems. The proposed framework is general enough for both regression and classification settings. Our empirical results show that the proposed approach explains nearly as much variation as PCA while outperforming existing methods in prediction accuracy.