Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds

Gael Gendron, Joze M. Rozanec, Michael J. Witbrock, Gillian Dobbie

Keywords: Causal Extraction, Causal Inference, Counterfactual Reasoning, Large Language Models, Compositionality, Explainability, Robustness, Agents, Retrieval-Augmented Generation, Efficiency

TL;DR: We introduce a framework extracting causal world knowledge and using it as compositional causal constraints for counterfactual reasoning, improving efficiency in real-world counterfactual inference.

Abstract:

Causal world models solve problems by decomposing them into high-level reusable causal mechanisms, allowing them to answer counterfactual questions about an environment of interest, i.e., predict how it would have evolved if an arbitrary subset of events had been realized differently. The ability to answer such questions is crucial for models to reliably understand the world under arbitrary distributions.
However, this task currently eludes large language models (LLMs), which do not have demonstrated causal reasoning capabilities beyond the memorization of existing causal relationships. Furthermore, evaluating counterfactuals in real-world applications is challenging since only the factual world is observed, limiting evaluation to synthetic datasets. We address these problems by proposing the Causal Cartographer, a twofold system composed of two agents: the first extracts causal relationships from data and builds a vast repository of compositional causal knowledge, while the second uses them as constraints to perform reliable step-by-step causal inference. We evaluate our approach on real-world counterfactuals obtained by matching data from diverse news sources. We show that our approach can extract accurate knowledge and enhance the robustness of LLMs for causal reasoning tasks. In particular, the proposed causal conditioning mitigates the impact of spurious correlations and greatly reduces inference cost (by up to 70\%) compared to chain-of-thought reasoning.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 25