Learning Linear Causal Representations from Interventions under General Nonlinear Mixing

Published: 21 Sept 2023, Last Modified: 21 Dec 2023NeurIPS 2023 oralEveryoneRevisionsBibTeX
Keywords: Causal Representation Learning, Interventional data, Gaussian Structural Causal models
TL;DR: We prove identifiability of causal representation learning from interventions with general nonlinear mixing functions and unknown, latent interventions, and propose a contrastive learning algorithm to learn it.
Abstract: We study the problem of learning causal representations from unknown, latent interventions in a general setting, where the latent distribution is Gaussian but the mixing function is completely general. We prove strong identifiability results given unknown single-node interventions, i.e., without having access to the intervention targets. This generalizes prior works which have focused on weaker classes, such as linear maps or paired counterfactual data. This is also the first instance of identifiability from non-paired interventions for deep neural network embeddings and general causal structures. Our proof relies on carefully uncovering the high-dimensional geometric structure present in the data distribution after a non-linear density transformation, which we capture by analyzing quadratic forms of precision matrices of the latent distributions. Finally, we propose a contrastive algorithm to identify the latent variables in practice and evaluate its performance on various tasks.
Supplementary Material: pdf
Submission Number: 11309