Representation Learning from Interventional Data

Gautam Sreekumar, Vishnu Boddeti

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: causal representation learning, representation learning, interventional data

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: How to leverage the available interventional data better with the help of the underlying causal graph

Abstract: To learn data representations that are robust to distribution shifts, practitioners conduct interventions and collect interventional data in addition to passively collected observational data. However, even when the underlying causal model is known, existing approaches treat interventional data like observational data and ignore the causal model. Furthermore, these approaches assume a large number of interventional data points obtained through interventions that span the entire support of the intervened variable. This leads to representations that exhibit large discrepancies in predictive performance on observational and interventional data. In this paper, we first identify a strong correlation between interventional performance and adherence of the features to the statistical independence conditions induced by the underlying causal model. Then, we exploit this correlation and propose RepLIn to explicitly enforce the statistical independence during interventions. We demonstrate the utility of RepLIn across representative image classification tasks (attribute prediction on CelebA and image classification under corruption on CIFAR-10C and ImageNet-C) by modeling them as causal graphs and learning representations that are more robust to interventional distribution shifts.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3823