Learning Invariant Representations with Missing DataDownload PDF

Published: 02 Dec 2021, Last Modified: 17 Sept 2023NeurIPS 2021 Workshop DistShift PosterReaders: Everyone
Keywords: invariant prediction, spurious correlations, shorcuts, missingness, missing data, inverse-weighted, doubly-robust, ipcw
TL;DR: Objectives for invariant prediction enforce independencies between models and nuisance variables. These objectives are hard to estimate under missingness. We propose a doubly-robust estimator for the MMD under nuisance missingness.
Abstract: Spurious correlations allow flexible models to predict well during training but poorly on related test populations. Recent work has shown that models that satisfy particular independencies involving correlation-inducing nuisance variables have guarantees on their test performance. Enforcing such independencies requires nuisances to be observed during training. However, nuisances, such as demographics or image background labels, are often missing. Enforcing independence on just the observed data does not imply independence on the entire population. Here we derive MMD estimators used for invariance objectives under missing nuisances. On simulations and clinical data, optimizing through these estimates achieves test performance similar to using estimators that make use of the full data.
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 2 code implementations](https://www.catalyzex.com/paper/arxiv:2112.00881/code)
1 Reply