Identifying Causal Direction via Variational Bayesian Compression

Quang-Duy Tran, Bao Duong, Phuoc Nguyen, Thin Nguyen

TL;DR: A framework leveraging variational Bayesian compression of neural networks for the cause-effect identification task

Abstract: Telling apart the cause and effect between two random variables with purely observational data is a challenging problem that finds applications in various scientific disciplines. A key principle utilized in this task is the algorithmic Markov condition, which postulates that the joint distribution, when factorized according to the causal direction, yields a more succinct codelength compared to the anti-causal direction. Previous approaches approximate these codelengths by relying on simple functions or Gaussian processes (GPs) with easily evaluable complexity, compromising between model fitness and computational complexity. To overcome these limitations, we propose leveraging the variational Bayesian learning of neural networks as an interpretation of the codelengths. Consequently, we can enhance the model fitness while promoting the succinctness of the codelengths, while avoiding the significant computational complexity of the GP-based approaches. Extensive experiments on both synthetic and real-world benchmarks in cause-effect identification demonstrate the effectiveness of our proposed method, surpassing the overall performance of related complexity-based and structural causal model regression-based approaches.

Lay Summary: Many scientific questions aim at answering cause-and-effect questions of what causes what—for example, does A cause B, or the other way around? Such questions are difficult to answer when we can only passively observe the data without any active interaction. Our research addresses this challenge through the principle of Occam's razor, where the simplest yet sufficiently accurate explanation of the data is likely the cause. Previous methods rely on either over-simplified or overly complex models, making a trade-off between accuracy and simplicity. Instead, we employ a more flexible approach based on Bayesian neural networks, which are tools that can capture complex patterns while still favoring simpler explanations. This provides us with a better balance between accuracy and simplicity, making it effective at telling apart the cause and effect. Results on both simulated and real-world data show that our method reliably outperforms existing techniques in answering cause-and-effect questions.

Link To Code: https://github.com/quangdzuytran/COMIC

Primary Area: General Machine Learning->Causality

Keywords: causal discovery, neural networks, variational Bayesian code

Submission Number: 6121