Key Qualities of Conversational Recommender Systems: From Users' Perspective

Yucheng Jin, Li Chen, Wanling Cai, Pearl Pu

Abstract: An increasing number of recommender systems enable conversational interaction to enhance the system’s overall user experience (UX). However, it is unclear what qualities of a conversational recommender system (CRS) are essential to determine the success of a CRS. This paper presents a model to capture the key qualities of conversational recommender systems and their related user experience aspects. Our model incorporates the characteristics of conversations (such as adaptability, understanding, response quality, rapport, humanness, etc.) in four major user experience dimensions of the recommender system: User Perceived Qualities, User Belief, User Attitudes, and Behavioral Intentions. Following the psychometric modeling method, we validate the combined metrics using the data collected from an online user study of a conversational music recommender system. The user study results 1) support the consistency, validity, and reliability of the model that identifies seven key qualities of a CRS; and 2) reveal how conversation constructs interact with recommendation constructs to influence the overall user experience of a CRS. We believe that the key qualities identified in the model help practitioners design and evaluate conversational recommender systems.