Unleashing Potential of Evidence in Knowledge-Intensive Dialogue Generation

Xianjie Wu, Jian Yang, Tongliang Li, Shiwei Zhang, Yiyang Du, Linzheng Chai, Di Liang, Zhoujun Li

Abstract: Incorporating external knowledge into dialogue generation (DG) is crucial for enhancing response accuracy, where evidence fragments serve as effective knowledgeable snippets that support factual dialogue replies. However, introducing irrelevant content beyond valid knowledge fragments can adversely affect reply quality and lead to hallucinated responses. Prior work relies on manual annotations to develop models for identifying evidence within external knowledge. However, these annotations often cover only a limited portion of the valid evidence, restricting the ability of models to mine useful evidence from retrieved knowledge. To fully Unleash the potential of evidence, we propose a framework to effectively incorporate Evidence in knowledge-Intensive Dialogue Generation (U-EIDG). Specifically, we develop an evidence miner (Evid-M) that harnesses the power of large language models (LLMs) to mine reliable evidence labels from external knowledge. Subsequently, we propose an evidence indicator (Evid-I) to effectively identify valid evidence from retrieved knowledge by utilizing these evidence labels. Furthermore, we introduce an evidence-augmented generator (EAG) incorporating an evidence-attention mechanism that enables the model to focus on segments supported by evidence. Experimental results on the MultiDoc2Dial and WoW benchmarks indicate that the proposed method significantly outperforms other baselines, with a +3∼5 points improvement in Rouge-L. Further analysis confirms the effectiveness of fully mining valid evidence fragments for knowledge-intensive dialogue generation.