BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation

Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, Rahul Gupta

Abstract:

Recent advances in deep learning techniques have enabled machines to generate cohesive open-ended text when prompted with
a sequence of words as context. While these models now empower
many downstream applications from conversation bots to automatic storytelling, they have been shown to generate texts that
exhibit social biases. To systematically study and benchmark social
biases in open-ended language generation, we introduce the Bias
in Open-Ended Language Generation Dataset (BOLD), a large-scale
dataset that consists of 23,679 English text generation prompts for
bias benchmarking across five domains: profession, gender, race,
religion, and political ideology. We also propose new automated
metrics for toxicity, psycholinguistic norms, and text gender polarity to measure social biases in open-ended text generation from
multiple angles. An examination of text generated from three popular language models reveals that the majority of these models
exhibit a larger social bias than human-written Wikipedia text
across all domains. With these results we highlight the need to
benchmark biases in open-ended language generation and caution
users of language generation models on downstream tasks to be
cognizant of these embedded prejudices.