R&R: A Role-playing Model Enhanced by Retrieving and Reflecting

ACL ARR 2025 May Submission3041 Authors

Abstract: Role-playing is a key capability of large language models (LLMs). However, existing models often fall short in delivering fully immersive character simulations. They frequently overlook the knowledge constraints inherent to the role and fail to adopt the appropriate mindset, resulting in responses that feel noticeably artificial. To address these limitations, we propose R&R, a role-playing model enhanced with retrieval and reflection. Prior to generating a response, our model first retrieves similar historical dialogues based on the current query and generates character-specific reflections informed by the role’s self-profile. It then searches for relevant background knowledge to support the response. Finally, the model evaluates whether the query falls within the character’s scope of knowledge and generates a response grounded in both the retrieved context and reflective reasoning. To assess the effectiveness of our approach, we construct a new benchmark dataset and introduce novel evaluation metrics tailored to character role-play. We also conduct comparisons using an established public metric. Experimental results show that our model achieves an average performance improvement of 8% over CharacterLLM.

Paper Type: Long

Research Area: Dialogue and Interactive Systems

Research Area Keywords: spoken dialogue systems, evaluation and metrics, retrieval, knowledge augmented, applications

Contribution Types: Data resources, Data analysis, Theory

Languages Studied: English, Chinese

Submission Number: 3041