CReTIHC: Designing Causal Reasoning Tasks about Temporal Interventions and Hallucinated Confoundings

Changwoo Chun, SongEun Lee, Jaehyung Seo, Heuiseok Lim

Submission Type: Regular Short Paper

Submission Track: Resources and Evaluation

Submission Track 2: Commonsense Reasoning

Keywords: Causality, Commonsense reasoning, Large Language Models, Temporal Interventions, Hallucinated Confoundings

TL;DR: CReTIHC is a novel dataset that enhances the causal reasoning of Large Language Models by emphasizing the distinction between true causal relationships and the noise introduced by temporal interventions and hallucinated confoundings.

Abstract: Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their ability to establish causal relationships, particularly in the context of temporal interventions and language hallucinations, remains challenging. This paper presents \textbf{CReTIHC}, a novel dataset designed to test and enhance the causal reasoning abilities of LLMs. The dataset is constructed using a unique approach that incorporates elements of verbal hallucinations and temporal interventions through the reengineering of existing causal inference datasets. This transformation creates complex scenarios that push LLMs to critically evaluate the information presented and identify cause-and-effect relationships. The CReTIHC dataset serves as a pioneering tool for improving LLM's causal inference capabilities, paving the way for a more nuanced understanding of causal relationships in natural language processing (NLP) tasks. The whole dataset is publicly accessible at: (https://github.com/ChangwooChun/CReTIHC)

Submission Number: 3553