Is Self-knowledge and Action Consistent or Not: Investigating Large Language Model's Personality
Abstract: In this study, we delve into the validity of conventional personality questionnaires in capturing the human-like personality traits of Large Language Models (LLMs). Our objective is to assess the congruence between the personality traits LLMs claim to possess and their demonstrated tendencies in real-world scenarios. By conducting an extensive examination of LLM outputs against observed human response patterns, we discover the disjunction between self-knowledge and action in LLMs. And we formulate hypotheses grounded in psychological theories and metrics, offering insights into the intricate mechanisms driving the observed discrepancy.
Paper Type: Long
Research Area: Linguistic theories, Cognitive Modeling and Psycholinguistics
Research Area Keywords: Computational psycholinguistics, Cognitive modeling, Evaluation methodologies
Contribution Types: Model analysis & interpretability, Data resources
Languages Studied: English, Chinese
Submission Number: 350
Loading