ICDA: Interactive Causal Discovery through Large Language Model Agents
Keywords: Causal Discovery, LLM, Black box optimizer
TL;DR: We utilize LLMs as a black box optimizer to iteratively propose and update interventions on a causal graph
Abstract: Large language models (\textbf{LLMs}) have emerged as a powerful method for causal discovery. Instead of utilizing numerical observational data, LLMs utilize associated variable \textit{semantic metadata} to predict causal relationships. Simultaneously, LLMs demonstrate impressive abilities to act as black-box optimizers when given an objective $f$ and sequence of trials. We study LLMs at the intersection of these two capabilities by applying LLMs to the task of \textit{interactive causal discovery}: given a budget of $I$ edge interventions over $R$ rounds, minimize the distance between the ground truth causal graph $G^*$ and the predicted graph $\hat{G}_R$ at the end of the $R$-th round. We propose an LLM-based pipeline incorporating two key components: 1) an LLM uncertainty-driven method for edge intervention selection 2) a local graph update strategy utilizing binary feedback from interventions to improve predictions for non-intervened neighboring edges. Experiments on eight different real-world graphs show our approach significantly outperforms a random selection baseline: at times by up to 0.5 absolute F1 score. Further we conduct a rigorous series of ablations dissecting the impact of each component of the pipeline. Finally, to assess the impact of memorization, we apply our interactive causal discovery strategy to a complex, new (as of July 2024) causal graph on protein transcription factors. Overall, our results show LLM driven uncertainy based edge selection with local updates performs strongly and robustly across a diverse set of real-world graphs.
Primary Area: causal reasoning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6871
Loading