Social Bias Evaluation for Large Language Models Requires Prompt Variations

ACL ARR 2025 February Submission3572 Authors

Abstract:

Warning: This paper contains examples of stereotypes and biases.
Large Language Models (LLMs) exhibit considerable social biases, and various studies have tried to evaluate and mitigate these biases accurately.
Previous studies use downstream tasks to examine the degree of social biases for evaluation and mitigation.
While the output of LLMs highly depends on prompts, prior works evaluating and mitigating bias have often relied on a limited variety of prompts.
In this paper, we investigate the sensitivity of LLMs when changing prompt variations (task instruction, few-shot examples, debias-prompt) by analyzing task performance and social bias of LLMs.
Our experimental results reveal LLM rankings fluctuate across prompts for both task performance and social bias.
We also confirmed that the impact of format changes can differ for each bias category.
Performance improvement from prompt settings may not result in reduced bias.
Moreover, the ambiguity of instances is a common factor in LLM sensitivity to prompts across advanced LLMs.
We recommend using diverse prompts, as in this study, to compare the effects of prompts on social bias in LLMs.
Our code will be publicly available after acceptance.

Paper Type: Long

Research Area: Ethics, Bias, and Fairness

Research Area Keywords: model bias/fairness evaluation

Contribution Types: Model analysis & interpretability, Position papers

Languages Studied: English

Submission Number: 3572