Exploring the Impact of Occupational Personas on Domain-Specific QA
Abstract: Recent studies on personas have improved the way Large Language Models (LLMs) interact with users, but the impact of personas on knowledge-based Question Answering (QA) tasks has been underexplored. Inspired by Holland Occupational Themes, this study proposes Profession-Based Personas (PBPs) and Occupational Personality-Based Personas (OPBPs) to enhance performance in domain-specific QA tasks. We investigate the impact of PBP and OPBP on scientific datasets within the Massive Multitask Language Understanding (MMLU) benchmark. Experimental results show that PBPs, exemplified by the "scientist", achieve an accuracy improvement of 1.29\% over the baseline. In contrast, the "artist" displays the lowest performance, with a 31.21\% decrease and significant variability. Our findings demonstrate that assigning PBPs to LLMs enhances models' ability to invoke domain knowledge. Additionally, we observed that OPBPs might lead to lower performance, even when the defined personality type is relevant to the task.
Paper Type: Short
Research Area: Generation
Research Area Keywords: analysis
Contribution Types: NLP engineering experiment
Languages Studied: English
Submission Number: 897
Loading