R&R: A Role-playing Model Enhanced by Retrieving and Reflecting

ACL ARR 2025 February Submission2916 Authors

Abstract: Role-playing is a crucial capability of large language models (LLMs). However, existing models often struggle to fully immerse users in character roles, as they typically fail to recognize the knowledge limitations expected within their roles and lack the appropriate mindset, making their performance noticeably artificial. To address these challenges, we propose R&R, a role-playing model enhanced by retrieval and reflection. Before generating responses, our model first matches similar historical dialogues based on the questions asked and generates reflections according to the role's self-profile. Then, the model searches for relevant knowledge to use in generating a response. Finally, our model assesses whether the question can be answered and generates a response based on the reflections and relevant knowledge. To evaluate the effectiveness of our model, we build a new dataset and propose new evaluation metrics. We also compare our model with others using a publicly available and validated evaluation metric. The results demonstrate that the average performance of our model improves by 8% over CharacterLLM, and human tests show that our model outperforms the others.

Paper Type: Long

Research Area: Dialogue and Interactive Systems

Research Area Keywords: spoken dialogue systems, evaluation and metrics, retrieval, knowledge augmented, applications

Contribution Types: Data resources, Data analysis, Theory

Languages Studied: English, Chinese

Submission Number: 2916