Don't Do That! Reverse Role Prompting Helps Large Language Models Stay in Personality Traits

Siyuan Chen, Pittawat Taveekitworachai, Yi Xia, Xiaoxu Li, Mustafa Can Gursesli, Antonio Lanatà, Andrea Guazzini, Ruck Thawonmas

Abstract: This paper investigates effectiveness of role prompting, an approach used to condition a large language model (LLM) with a role or personality trait. Conditioning an LLM with a specific role or personality trait is crucial for various applications, such as using an LLM as a non-playable character. Role-playing is one of the crucial techniques in doing so. However, existing studies have only observed changes when introducing a role or personality trait to LLMs, but not how effectively they conform to the given personality trait. In this study, we investigate how well LLMs adhere to given instructions using a well-established personality test. Additionally, we conduct an experiment to determine how a given personality influences biases in generating a game story ending. Through our investigations, we found that traditional role prompting is ineffective for assessing a certain personality trait. Therefore, we propose a novel variant of role prompting called reverse role prompting (RRP) to reduce the significance of personality traits other than the assigned personality. We observed that when using RRP to assign personalities to LLMs, the LLMs are better able to act the given personality. We also found that LLMs inherently have high agreeableness, affecting story ending biases when instructed to generate a game story. RRP is also effective for reducing the effects of this inherent agreeableness. Future studies should further investigate the effectiveness of the proposed RRP for a wider range of narrative elements and applications in storytelling.