Disentangled Representation Learning in Non-Markovian Causal Systems

Adam Li, Yushu Pan, Elias Bareinboim

Keywords: causal representation learning, disentanglement, nonlinear ICA

TL;DR: We introduce a causal representation identifiability algorithm for determining which latent variables are disentangleable given a homogenous set of input distributions from multiple domains and knowledge of the latent variable causal structure.

Abstract:

Considering various data modalities, such as images, videos, and text, humans perform causal reasoning using high-level causal variables, as opposed to operating at the low, pixel level from which the data comes.
In practice, most causal reasoning methods assume that the data is described as granular as the underlying causal generative factors, which is often violated in various AI tasks.
This mismatch translates into a lack of guarantees in various tasks such as generative modeling, decision-making, fairness, and generalizability, to cite a few.
In this paper, we acknowledge this issue and study the problem of causal disentangled representation learning from a combination of data gathered from various heterogeneous domains and assumptions in the form of a latent causal graph. To the best of our knowledge, the proposed work is the first to consider i) non-Markovian causal settings, where there may be unobserved confounding, ii) arbitrary distributions that arise from multiple domains, and iii) a relaxed version of disentanglement. Specifically, we introduce graphical criteria that allow for disentanglement under various conditions. Building on these results, we develop an algorithm that returns a causal disentanglement map, highlighting which latent variables can be disentangled given the combination of data and assumptions. The theory is corroborated by experiments.

Primary Area: Causal inference

Submission Number: 13821