Consistency Is the Key: Detecting Hallucinations in LLM Generated Text By Checking Inconsistencies About Key Facts

ACL ARR 2025 February Submission7839 Authors

Abstract: Large language models (LLMs), despite their remarkable text generation capabilities, often hallucinate and generate text that is factually incorrect and not grounded in real-world knowledge. Such hallucinations are particularly concerning in domains such as healthcare, finance, and customer support, where incorrect information can have severe consequences. A typical way to use LLMs is via the APIs provided by LLM vendors where there is no access to model weights or options to fine-tune the model to control its behavior. Existing methods to detect hallucinations in settings where the model access is restricted or constrained by resources typically require making multiple calls to the underlying LLM to check and refine the output or to sample many responses to estimate output probabilities. Such a large number of calls significantly increases latency and cost, becoming a bottleneck for adopting these methods in practical scenarios. We introduce CONFACTCHECK, an efficient hallucination detection approach that does not leverage any external knowledge base and works on the simple intuition that responses to questions probing factual components of the text should be consistent within a single LLM and across different LLMs. Rigorous empirical evaluation on multiple datasets that cover both the generation of factual texts and the open generation reveals the strengths of CONFACTCHECK compared to the state-of-the-art baselines. CONFACTCHECK can detect hallucinated facts efficiently using fewer resources and achieves significantly higher accuracy scores compared to existing baselines that operate under similar conditions. We will release the code and resources on acceptance.

Paper Type: Long

Research Area: Generation

Research Area Keywords: Efficient/Low-Resource Methods for NLP, Ethics, Bias, and Fairness, Generation, Language Modeling

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 7839