Exploring In-Context Learning for Knowledge Grounded Dialog Generation

Qinyu Chen, Wenhao Wu, Sujian Li

Submission Type: Regular Long Paper

Submission Track: Dialogue and Interactive Systems

Submission Track 2: Commonsense Reasoning

Keywords: dialog, knowledge, large language models, in-context learning, retrieval system

Abstract:

Large neural-based dialog generation models have been applied in many real-life scenarios, yet they are prone to hallucination and tend to produce factually inaccurate outputs which raise great concerns.
To alleviate this problem, we propose a plug-and-play retrieval-based framework IKA, which leverages in-context learning and retrieval techniques to enhance LLMs on knowledge grounded dialog generation.
We design thorough experiments on a large-scale knowledge graph with 1M+ facts to investigate the effectiveness and generalization of our framework.
Experiments show that our method surpasses previous training-based SOTA by a large margin, specifically 46.67% in BLEU4, 26.01% in ROUGE-L, 122.90% in BARTScore and 30.50% in Entity Coverage F1.
Further analysis show promising abilities of LLMs to perform knowledge-intensive tasks, which is previously considered weak and understudied.

Submission Number: 3601