Deep Autoregressive Models as Causal Inference Engines

Daniel Jiwoong Im, Kevin Zhang, Nakul Verma, Kyunghyun Cho

Abstract: Existing causal inference (CI) models are often restricted to data with low-dimensional confounders and singleton actions. We propose an autoregressive (AR) CI framework capable of handling complex confounders and sequential actions commonly found in modern applications. Our approach accomplishes this using sequencification, which transforms data from an underlying causal diagram into a sequence of tokens. Sequencification not only accommodates training with data generated from a large class of DAGs, but also extends existing CI capabilities to estimate multiple causal quantities using a single model. We can directly compute probabilities from interventional distributions, simplifying inference and improving outcome prediction accuracy. We demonstrate that an AR model adapted for CI is efficient and effective in various complex applications such as navigating mazes, playing chess endgames, and evaluating the impact of certain keywords on paper acceptance rates, where we consider causal queries beyond standard reinforcement learning-type questions.

Submission Length: Regular submission (no more than 12 pages of main content)

Code: https://github.com/jiwoongim/Deep-Autoregressive-Models-as-Causal-Inference-Engines

Assigned Action Editor: ~Mingming_Gong1

Submission Number: 4542