Causal Representation Learning and Inference for Generalizable Cross-Domain Predictions

Naiyu Yin, Hanjing Wang, Amit Dhurandhar, Tian Gao, Qiang Ji

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Generalizable representation learning; Causal Intervention

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose a causal representation learning framework based on a novel SCM

Abstract:

Learning generalizable representations for machine learning and computer vision tasks is an active area of research. Typically, methods utilize data from multiple domains and seek to transfer the invariant representations to new and unseen domains. This paper proposes to perform causal inference on transportable, invariant interventional distribution to improve the prediction performance under distribution shifts.
Specifically, we first introduce a structural causal model (SCM) with latent representations to capture the underlying causal mechanism that underpins the data generation process. Subject to the proposed SCM model, we can perform the intervention on the spurious representations that are affected by domain-specific factors and the latent confounders to eliminate the spurious correlations. Guided by the proposed SCM and the invariant interventional distribution, we propose a causal representation learning framework. Compared to state-of-the-art domain generalization approaches, our method is robust and generalizable under distribution shifts. Furthermore, the empirical study shows that the proposed causal representation scheme outperforms existing causal learning baselines.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2044