A Formal Verification Framework for LLM-Generated Causal Expressions

ACL ARR 2025 May Submission2563 Authors

Abstract: Causal reasoning is a critical aspect in human judgment, yet large language models (LLMs) often fail to distinguish between correlation and causation when answering natural language questions. While existing benchmarks primarily assess factual correctness in causal QA tasks, they do not evaluate whether model outputs are causally coherent or formally valid. In this work, we propose a symbolic evaluation framework that assesses whether LLM-generated answers can be correctly formalized as causal expressions using do-calculus semantics. Our approach translates LLM outputs into structured forms such as $P(Y\mid \text{do}(X), Z)$, and compares them against known ground-truth assumptions or causal graphs, and identifies common reasoning failures such as misinterpreting interventions as observations. We further demonstrate that this formalization layer enables symbolic feedback, which can guide LLMs to revise incorrect outputs and improve overall answer quality.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: evaluation methodologies, metrics

Contribution Types: Model analysis & interpretability, Publicly available software and/or pre-trained models, Data resources, Theory

Languages Studied: English

Submission Number: 2563