Leveraging Ensemble Techniques and Metadata for Subjective Knowledge-grounded Conversational Systems

Seongho Joo, Kang-il Lee, Kyungmin Min, Joongbo Shin, Janghoon Han, Seungpil Won, Kyomin Jung

Abstract:

The goal of DSTC11 track 5 is to build taskoriented dialogue systems that can effectively utilize external knowledge sources such as
FAQs and reviews. This year’s challenge differs from previous ones as it includes subjective knowledge snippets and requires multiple snippets for a single turn. We propose a pipeline system for the challenge focusing on entity tracking, knowledge selection and response generation. Specifically, we devise a novel heuristic to ensemble the outputs from the rule-based method and neural model for entity tracking and knowledge selection. We also leverage metadata information in the knowledge source to handle fine-grained user queries. Our approach achieved the first place in objective evaluation and the third place in human evaluation of DSTC11 track 5