Lightweight Query Checkpoint: Classifying Faulty User Queries to Mitigate Hallucinations in Large Language Model Question Answering

ACL ARR 2025 February Submission8267 Authors

Abstract: Question Answering (QA) with large language models has shown impressive performance, yet hallucinations still persist, particularly when user queries carry incorrect premises, insufficient context, or linguistic ambiguity. To address this issue, we propose \textbf{Lightweight Query Checkpoint (\textsc{Lqc})}, a small classification model that detects verification-required queries before the LLM generates a potentially faulty answer. \textsc{Lqc} leverages hidden states extracted from intermediate layers of a smaller-scale, non-instruct-tuned LLM to effectively distinguish queries requiring verification from clear queries. We first systematically define categories of queries that need verification, construct a dataset comprising both defective and clear queries, and train a binary contrastive learning model. Through extensive experiments on various QA datasets, we demonstrate that incorporating \textsc{Lqc} into QA pipelines reduces hallucinations while preserving strong answer quality.

Paper Type: Long

Research Area: Question Answering

Research Area Keywords: Question Answering, NLP Applications, Efficient/Low-Resource Methods for NLP

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 8267