Mitigating Biases in Hate Speech Detection from A Causal Perspective

Zhehao Zhang, Jiaao Chen, Diyi Yang

Submission Type: Regular Long Paper

Submission Track: Computational Social Science and Cultural Analytics

Submission Track 2: Ethics in NLP

Keywords: Hate speech detection, Causal inference, Bias mitigation

TL;DR: We analyze biases in hate speech detection from a causal perspective and propose a new metric and method for bias mitigation based on two confounders.

Abstract: Nowadays, many hate speech detectors are built to automatically detect hateful content. However, their training sets are sometimes skewed towards certain stereotypes (e.g., race or religion-related). As a result, the detectors are prone to depend on some shortcuts for predictions. Previous works mainly focus on token-level analysis and heavily rely on human experts' annotations to identify spurious correlations, which is not only costly but also incapable of discovering higher-level artifacts. In this work, we use grammar induction to find grammar patterns for hate speech and analyze this phenomenon from a causal perspective. Concretely, we categorize and verify different biases based on their spuriousness and influence on the model prediction. Then, we propose two mitigation approaches including Multi-Task Intervention and Data-Specific Intervention based on these confounders. Experiments conducted on 9 hate speech datasets demonstrate the effectiveness of our approaches.

Submission Number: 3186