Irrelevant Alternatives Bias Large Language Model Hiring Decisions
Abstract: We investigate whether LLMs display a well-known human cognitive bias, the attraction effect, in hiring decisions.
The attraction effect occurs when the presence of an inferior candidate makes a superior candidate more appealing, increasing the likelihood of the superior candidate being chosen over a non-dominated competitor.
Our study finds consistent and significant evidence of the attraction effect in GPT-3.5 and GPT-4 when they assume the role of a recruiter.
Irrelevant attributes of the decoy, such as its gender, further amplify the observed bias. GPT-4 exhibits greater bias variation than GPT-3.5.
Our findings remain robust even when warnings against the decoy effect are included and the recruiter role definition is varied.
Paper Type: Long
Research Area: Ethics, Bias, and Fairness
Research Area Keywords: model bias/fairness evaluation, ethical considerations in NLP applications
Contribution Types: NLP engineering experiment
Languages Studied: English
Submission Number: 1474
Loading