Closed-form supervised dimensionality reduction with generalized linear models

Irina Rish, Genady Grabarnik, Guillermo A. Cecchi, Francisco Pereira, Geoffrey J. Gordon

Abstract: We propose a family of supervised dimensionality reduction (SDR) algorithms that combine feature extraction (dimensionality reduction) with learning a predictive model in a unified optimization framework, using data- and class-appropriate generalized linear models (GLMs), and handling both classification and regression problems. Our approach uses simple closed-form update rules and is provably convergent. Promising empirical results are demonstrated on a variety of high-dimensional datasets.