ChatCRS: Incorporating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems

Anonymous

Abstract: This paper aims to efficiently enable Large Language Models (LLMs) to use external knowledge and goal guidance in conversational recommendation system (CRS) tasks. Advanced LLMs (e.g., ChatGPT) are limited in CRS tasks for 1) generating grounded responses with recommendation-oriented knowledge, or 2) proactively guiding users through different dialogue goals. In this work, we first analyze those limitations through a comprehensive evaluation to assess LLMs' intrinsic capabilities, showing the necessity of incorporating external knowledge and goal guidance which contribute significantly to the recommendation accuracy and language quality. In light of this finding, we propose a novel ChatCRS framework to decompose the complex CRS task into several sub-tasksthrough the implementation of 1) a knowledge retrieval agent using a tool-augmented approach to reason over external Knowledge Bases and 2) a goal-planning agent for dialogue goal prediction. Experimental results on two CRS datasets reveal that ChatCRS sets new state-of-the-art benchmarks, improving language quality of informativeness by 17% and proactivity by 27%, and achieving a tenfold enhancement in recommendation accuracy over LLM-based CRS.

Paper Type: long

Research Area: Dialogue and Interactive Systems

Contribution Types: Model analysis & interpretability, Approaches to low-resource settings

Languages Studied: English, Chinese