Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generation
Keywords: hallucination, semantic certainty, natural language generation, knowledge grounded dialogue generation
Abstract: Model hallucination has been a crucial interest of research in Natural Language Generation (NLG).
In this work, we propose sequence-level certainty as a common theme over hallucination in NLG, and explore the correlation between sequence-level certainty and the level of hallucination in model responses.
We categorize sequence-level certainty into two aspects: probabilistic certainty and semantic certainty, and reveal through experiments on Knowledge-Grounded Dialogue Generation (KGDG) task that both a higher level of probabilistic certainty and a higher level of semantic certainty in model responses are significantly correlated with a lower level of hallucination.
What's more, we provide theoretical proof and analysis to show that semantic certainty is a good estimator of probabilistic certainty, and therefore has the potential as an alternative to probability-based certainty estimation in black-box scenarios.
Based on the observation on the relationship between certainty and hallucination, we further propose Certainty-based Response Ranking (CRR), a decoding-time method for mitigating hallucination in NLG.
Based on our categorization of sequence-level certainty, we propose $2$ types of CRR approach: Probabilistic CRR (P-CRR) and Semantic CRR (S-CRR).
P-CRR ranks individually sampled model responses using their arithmetic mean log-probability of the entire sequence.
S-CRR approaches certainty estimation from meaning-space, and ranks a number of model response candidates based on their semantic certainty level, which is estimated by the entailment-based Agreement Score (AS).
Through extensive experiments across $3$ KGDG datasets, $3$ decoding methods, and on $4$ different models, we validate the effectiveness of our $2$ proposed CRR methods to reduce model hallucination.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1010
Loading