Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model

ACL ARR 2024 December Submission1711 Authors

Abstract:

The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. However, current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases. (II) The model learns to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level.
In this work, we propose framework called Beyond Dialogue, designed to overcome these hurdles. This framework introduces "beyond dialogue" tasks to align dialogue with profile traits based on each specific scenario, thereby eliminating biases during training. Furthermore, by adopting an innovative prompting mechanism that generates reasoning outcomes for training, the framework allows the model to achieve fine-grained alignment between profile and dialogue at the sentence level.
The aforementioned methods are fully automated and low-cost.
Experimental results show that our model excels in adhering to and reflecting various dimensions of role profiles, outperforming most proprietary general and specialized role-playing baselines.

Paper Type: Long

Research Area: Language Modeling

Research Area Keywords: applications, fine-tuning, prompting

Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models, Data resources

Languages Studied: English, Chinese

Submission Number: 1711