Task-Relevant Feature Selection with Prediction Focused Mixture Models

Abhishek Sharma, Catherine Zeng, Sanjana Narayanan, Sonali Parbhoo, Roy H. Perlis, Finale Doshi-Velez

Abstract:

Probabilistic models, such as mixture models, can encode latent structures that both explain the data and aid specific downstream tasks.
We focus on a constrained setting where we want to learn a model with relatively few components (e.g. for interpretability).
Simultaneously, we ensure that the components are useful for downstream predictions by introducing \emph{prediction-focused} modeling for mixtures, which automatically selects data features relevant to a prediction task.
Our approach identifies task-relevant input features, outperforms models that are not prediction-focused, and is easy to optimize; most importantly, we also characterize \emph{when} prediction-focused modeling can be expected to work.

Submission Length: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Aditya_Menon1

Submission Number: 1919