EVER: Mitigating Hallucination in Large Language Models through Generation-Time Verification and Rectification
Abstract: Large Language Models (LLMs) have demonstrated remarkable proficiency in generating fluent text. However, they often encounter the challenge of generating inaccurate or hallucinated content. This issue is common in both non-retrieval-based generation and retrieval-augmented generation approaches, and existing post-hoc rectification methods may not address the accumulated hallucination errors that may be caused by the "snowballing" issue, especially in reasoning tasks. To tackle these challenges, we introduce a novel approach called Generation-Time Verification and Rectification (EVER). Instead of waiting until the end of the generation process to rectify hallucinations, EVER employs a generation-time, step-wise generation and hallucination rectification strategy. Apart from directly mitigating hallucination, we further demonstrate that both the EVER-rectified response and the original one can serve as preference data to enhance the factuality of the model through preference tuning. When compared to both retrieval-based and non-retrieval-based baselines, EVER demonstrates a significant improvement in generating trustworthy and factually accurate text across a diverse range of tasks, including biography generation and multi-hop reasoning.
Paper Type: Long
Research Area: Syntax: Tagging, Chunking and Parsing
Research Area Keywords: Large Language Models, Hallucination
Languages Studied: English
Submission Number: 779
Loading