Integrating Large Language Models in Causal Discovery: A Statistical Causal Approach

MASAYUKI TAKAYAMA, Tadahisa OKUDA, Thong Pham, Tatsuyoshi Ikenoue, Shingo Fukuma, Shohei Shimizu, Akiyoshi Sannai

Abstract:

In practical statistical causal discovery (SCD), embedding domain expert knowledge as constraints into the algorithm is important for reasonable causal models reflecting the broad knowledge of domain experts, despite the challenges in the systematic acquisition of background knowledge.
To overcome these challenges, this paper proposes a novel method for causal inference, in which SCD and knowledge-based causal inference (KBCI) with a large language model (LLM) are synthesized through ``statistical causal prompting (SCP)'' for LLMs and prior knowledge augmentation for SCD.
The experiments in this work have revealed that the results of LLM-KBCI and SCD augmented with LLM-KBCI approach the ground truths, more than the SCD result without prior knowledge.
These experiments have also revealed that the SCD result can be further improved if the LLM undergoes SCP.
Furthermore, with an unpublished real-world dataset, we have demonstrated that the background knowledge provided by the LLM can improve the SCD on this dataset, even if this dataset has never been included in the training data of the LLM.
For future practical application of this proposed method across important domains such as healthcare, we also thoroughly discuss the limitations, risks of critical errors, expected improvement of techniques around LLMs, and realistic integration of expert checks of the results into this automatic process, with SCP simulations under various conditions both in successful and failure scenarios.
The careful and appropriate application of the proposed approach in this work, with improvement and customization for each domain,
can thus address challenges such as dataset biases and limitations, illustrating the potential of LLMs to improve data-driven causal inference across diverse scientific domains.

The code used in this work is publicly available at: https://github.com/mas-takayama/LLM-and-SCD.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=rJRmspwYuW&nesting=2&sort=date-desc

Changes Since Last Submission: Camera-Ready Version

Code: https://github.com/mas-takayama/LLM-and-SCD

Assigned Action Editor: ~Fabio_Stella1

Submission Number: 4060