Zero-Shot Learning of Causal Models

Divyat Mahajan, Jannes Gladrow, Agrin Hilmkil, Cheng Zhang, Meyer Scetbon

Keywords: Causality, Transformers, Generative Models

TL;DR: We propose an approach to amortize the learning of causal models, thus enabling zero shot inference.

Abstract:

With the increasing acquisition of datasets over time, we now have access to precise and varied descriptions of the world, capturing all sorts of phenomena.
These datasets can be seen as empirical observations of unknown causal generative processes, or Structural Causal Models (SCMs).
Recovering these causal generative processes from observations poses formidable challenges, and often require to learn a specific generative model for each dataset.
In this work, we propose to learn a \emph{single} model capable of inferring in a zero-shot manner the causal generative processes of datasets.
Rather than learning a specific SCM for each dataset, we enable FiP, the architecture proposed in~\cite{scetbon2024fip}, to infer the generative SCMs conditionally on their empirical representations.
More specifically, we propose to amortize the learning of a conditional version of FiP to infer directly the generative SCMs from observations and causal structures on synthetically generated datasets.
We show that our model is capable of predicting in zero-shot the true generative SCMs, and as a by-product, of (i) generating new dataset samples, and (ii) inferring intervened ones.
Our experiments demonstrate that our amortized procedure achieves performances on par with SoTA methods trained specifically for each dataset on both in and out-of-distribution problems.
To the best of our knowledge, this is the first time that SCMs are inferred in a zero-shot manner from observations, paving the way for a paradigmatic shift towards the assimilation of causal knowledge across datasets.

Primary Area: causal reasoning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2452