BiasX: “Thinking Slow” in Toxic Content Moderation with Explanations of Implied Social Biases

Yiming Zhang, Sravani Uttara Nanduri, Liwei Jiang, Tongshuang Wu, Maarten Sap

Submission Type: Regular Short Paper

Submission Track: Human-Centered NLP

Submission Track 2: NLP Applications

Keywords: Social biases, Toxicity moderation, Human-AI collaboration, Free-text explanations

Abstract: Toxicity annotators and content moderators often default to mental shortcuts when making decisions. This can lead to subtle toxicity being missed, and seemingly toxic but harmless content being over-detected. We introduce BiasX, a framework that enhances content moderation setups with free-text explanations of statements' implied social biases, and explore its effectiveness through a large-scale crowdsourced user study. We show that indeed, participants substantially benefit from explanations for correctly identifying subtly (non-)toxic content. The quality of explanations is critical: imperfect machine-generated explanations (+2.4% on hard toxic examples) help less compared to expert-written human explanations (+7.2%). Our results showcase the promise of using free-text explanations to encourage more thoughtful toxicity moderation.

Submission Number: 2087