Go to NeurIPS 2025 Workshop CogInterp homepageDo Large Language Models Show Biases in Causal Learning? Insights from Contingency Judgment
Keywords: large language models, illusions of causality, contingency judgement
TL;DR: In this paper, we investigate the extent to which state-of-the-art LLMs exhibit the illusion of causality when faced with a classic cognitive science paradigm: the contingency judgment task.
Abstract: Causal learning is the cognitive process of developing the capability of making causal inferences based on available information, often guided by normative princi- ples. This process is prone to errors and biases, such as the illusion of causality, in which people perceive a causal relationship between two variables despite lacking supporting evidence. This cognitive bias has been proposed to underlie many societal problems, including social prejudice, stereotype formation, misinforma- tion, and superstitious thinking. In this work, we examine whether large language models are prone to developing causal illusions when faced with a classic cog- nitive science paradigm: the contingency judgment task. To investigate this, we constructed a dataset of 1,000 null contingency scenarios (in which the available information is not sufficient to establish a causal relationship between variables) within medical contexts and prompted LLMs to evaluate the effectiveness of po- tential causes. Our findings show that all evaluated models systematically inferred unwarranted causal relationships, revealing a strong susceptibility to the illusion of causality. While there is ongoing debate about whether LLMs genuinely “under- stand” causality or merely reproduce causal language without true comprehension, our findings support the latter hypothesis and raise concerns about the use of lan- guage models in domains where accurate causal reasoning is essential for informed decision-making.
Submission Number: 36
Loading