Invariance & Causal Representation Learning: Prospects and Limitations

Simon Bing, Jonas Wahl, Urmi Ninad, Jakob Runge

Keywords: causal representation learning, invariance, invariant prediction, distributional robustness, identifiability, out-of-distribution prediciton

TL;DR: We investigate to which extent predictive invariance can be utilized for causal representation learning, presenting first impossibility results outlining the need for further constraints to move forward.

Abstract: In causal models, a given mechanism is assumed to be invariant to changes of other mechanisms. While this principle has been utilized for inference in settings where the causal variables are observed, theoretical insights when the variables of interest are latent are largely missing. We assay the connection between invariance and causal representation learning by establishing impossibility results which show that invariance alone is insufficient to identify latent causal variables. Together with practical considerations, we use these theoretical findings to highlight the need for additional constraints in order to identify representations by exploiting invariance.

Submission Number: 6