Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems
Submission Type: Regular Long Paper
Submission Track: Dialogue and Interactive Systems
Submission Track 2: Human-Centered NLP
Keywords: conversational recommender systems, mixed-initiative conversational agents, mixed-type conversational dataset
TL;DR: This paper introduces SalesOps, a framework for enhancing conversational recommender systems by incorporating educational value, and evaluates its effectiveness through a human study comparing SalesBot against professional salespeople.
Abstract: Making big purchases requires consumers to research or consult a salesperson to gain domain expertise. However, existing conversational recommender systems (CRS) often overlook users' lack of background knowledge, focusing solely on gathering preferences.
In this work, we define a new problem space for conversational agents that aim to provide both product recommendations and educational value through mixed-type mixed-initiative dialog.
We introduce SalesOps, a framework that facilitates the simulation and evaluation of such systems by leveraging recent advancements in large language models (LLMs).
We build SalesBot and ShopperBot, a pair of LLM-powered agents that can simulate either side of the framework.
A comprehensive human study compares SalesBot against professional salespeople, revealing that although SalesBot approaches professional performance in terms of fluency and informativeness, it lags behind in recommendation quality. We emphasize the distinct limitations both face in providing truthful information, highlighting the challenges of ensuring faithfulness in the CRS context. We release our code and make all data available.
Submission Number: 2197
Loading