Developing an occupational prestige scale using Large Language Models

Robert de Vries, Mark J. Hill, Laura Ruis

Track: Technical

Keywords: societal bias; occupational prestige; social stratification; LLM

TL;DR: We propose a method for extracting rankings of occupational prestige from an LLM, validate it against existing human data, and discuss benefits of using LLMs for social stratification research.

Abstract: Large Language Models (LLMs), being trained on large fractions of all online text, reflect societal biases and stereotypes – such as racial and gender biases. In this paper, we propose a method of using such models to capture societal perceptions of occupational prestige. We create four occupational prestige scales using this method, with each tapping a difference facet of prestige perceptions. These scales are validated against existing prestige scales based on human data. We conclude that it is possible to create valid measures of occupational prestige by prompting commercially available LLMs – though with some important limitations. Implications for future social stratification research are discussed.

Submission Number: 12