Biasly: An Expert-Annotated Dataset for Subtle Misogyny Detection and Mitigation

Anonymous

TL;DR: We introduce expert-annotated Biasly dataset on misogyny in North American films for enhanced NLP bias detection, explanation, and removal.

Abstract: Using novel approaches to dataset development, the Biasly dataset captures the nuance and subtlety of misogyny in ways that are unique within the literature. Built in collaboration with multi-disciplinary experts and annotators themselves, the dataset contains annotations of movie subtitles, capturing colloquial expressions of misogyny in North American film. The open-source dataset can be used for a range of NLP tasks, including binary and multi-label classification, severity score regression, and text generation for rewrites. In this paper, we discuss the methodology used, analyze the annotations obtained, provide baselines for each task using common NLP algorithms, and furnish error analyses to give insight into model behaviour when fine-tuned on the Biasly dataset.

Paper Type: long

Research Area: Ethics, Bias, and Fairness

Contribution Types: Data resources, Data analysis

Languages Studied: English