Detecting Dialogue Hallucination Using Graph Neural Networks

Kazuaki Furumai, Yanan Wang, Makoto Shinohara, Kazushi Ikeda, Yi Yu, Tsuneo Kato

Abstract: Even though large language models (LLMs) accumulate tremendous knowledge, dialogue systems built with LLMs induce hallucinations, leading to the generation of non-factual responses. How to provide proper references to achieve interpretable hallucination detection is a key issue that needs to be addressed. In this paper, we propose a graph neural network (GNN)-based method to achieve high-performance and interpretable hallucination detection for domain-specific dialogue systems. The method involves performing graph matching between a reference knowledge graph obtained from a knowledge database and a response knowledge graph extracted from the response to detect non-factual responses. By comparing with strong baselines, our method achieves a recall improvement of up to 11% and infers the cause of hallucinations with a probability of over 79%.