On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?

Anonymous

Paper Link: https://openreview.net/forum?id=Dgqw0N7BNli

Paper Type: Short paper (up to four pages of content + unlimited references and appendices)

Abstract: Knowledge-grounded conversational models are known to suffer from producing factually invalid statements, a phenomenon commonly called hallucination. In this work, we investigate the underlying causes of this phenomenon: is hallucination due to the training data, or to the models? We conduct a comprehensive human study on both existing knowledge-grounded conversational benchmarks and several state-of-the-art models. Our study reveals that the standard benchmarks consist of > 60% hallucinated responses, leading to models that not only hallucinate but even amplify hallucinations. Our findings raise important questions on the quality of existing datasets and models trained using them. We make our annotations publicly available for future research.

Presentation Mode: This paper will be presented in person in Seattle

Copyright Consent Signature (type Name Or NA If Not Transferrable): Nouha Dziri

Copyright Consent Name And Address: University of Alberta/ Mila