PrimeX: A Dataset of Worldview, Opinion, and Explanation

Rik Koncel-Kedziorski, Brihi Joshi, Tim Paek

Keywords: human-centered evaluation, human factors in NLP, cognitive modeling, psycho-demographic trait prediction

TL;DR: A novel dataset for personalization and psychological research consisting of public opinion survey responses with written explanations and a Primal World Beliefs survey for 885 respondents.

Abstract: As the adoption of language models advances, so does the need to better represent individual users to the model. Are there aspects of an individual's belief system that a language model can utilize for improved alignment? Following prior research, we investigate this question in the domain of opinion prediction by developing PrimeX, a dataset of public opinion survey data from 885 US residents with two additional sources of belief information: written explanations from the respondents for why they hold specific opinions, and the Primal World Belief survey for assessing respondent worldview. We provide an extensive initial analysis of our data and show the value of belief explanations and worldview for personalizing language models. Our results demonstrate how the additional belief information in PrimeX can benefit both the NLP and psychological research communities and open up avenues for further study.

Submission Type: Non-Archival

Paper Link: https://openreview.net/forum?id=9XLaK2ZTwt

Submission Number: 1