HALLUCHECK: An Efficient & Effective Fact-Based Approach Towards Factual Hallucination Detection Of LLMs Through Self-Consistency
Abstract: Large language models (LLMs) frequently generate inaccurate responses -- this can be particularly dangerous in sensitive areas like medicine and healthcare. Current methods for detecting hallucinations involve sampling answers multiple times, making them computationally intensive. In this study, we introduce HalluCheck, a novel hallucination detection module that identifies factual elements or atomic facts within a text. HalluCheck operates on the premise that responses to questions probing factual answers should be consistent both within a single LLM and across different LLMs. To improve system robustness, we incorporate a token-probability-based double-check mechanism. For hallucinated facts, inconsistencies or a lack of model confidence during generation will be evident. We evaluate our detection module on fact-based datasets such as NQ\_Open, HotpotQA, and WebQ, by building upon open-source LLMs such as LLaMa-2 (7B)-Instruct and Mistral-7B-Instruct. Finally, we compare the generated output with the correct answers to determine sentence-level AUC-ROC scores for hallucination detection. Our results demonstrate that HalluCheck can (i) detect hallucinated facts and (ii) achieve significantly higher AUC-ROC scores compared to existing baselines that operate under similar conditions, specifically those that do not utilize external databases for hallucination detection.
Paper Type: Long
Research Area: Generation
Research Area Keywords: Efficient/Low-Resource Methods for NLP, Ethics, Bias, and Fairness, Generation, Language Modeling
Contribution Types: Model analysis & interpretability, NLP engineering experiment
Languages Studied: English
Submission Number: 5745
Loading