Diversity-enhanced conversational recommendation via multi-agent reinforcement learning

Zihan Wang, Shi Feng, Daling Wang, Kaisong Song, Gang Wu, Yifei Zhang, Han Zhao, Ge Yu

Abstract: Multi-round conversational recommendation (MRCR) system assists users in finding the items they need with the fewest dialogue rounds by inquiring about desired features or making tailored recommendations. Numerous models employ single-agent Reinforcement Learning (RL) to accomplish MRCR and improve recommendation accuracy. However, they overlook the diversity of conversational recommendations and primarily focus on popular features or items. It impacts the fair visibility of the items and results in an unbalanced user experience. We propose a diversity-enhanced conversational recommendation model (DECREC), which is built on our proposed multi-agent RL framework. Compared to single-agent methods, this collaborative approach enables broader exploration of the action space, leading to more diverse decisions and recommendation results. Furthermore, we introduce a dynamic experience replay method that balances long-tail and head data. It ensures that each learning batch includes long-tail samples, keeping the model attentive to these less common but important data. Moreover, we incorporate feature entropy into the value estimation process, encouraging broader feature exploration and ultimately enhancing recommendation diversity. Extensive experiments on four public datasets demonstrate that DECREC reduces bias in MRCR and achieves optimal recommendation diversity and accuracy. Our code is available at https://github.com/wzhwzhwzh0921/DECREC.

External IDs:dblp:journals/kais/WangFWSWZZY25