Mitigate Gender Bias Using Negative Multi-task Learning

Liyuan Gao, Huixin Zhan, Victor S. Sheng

Abstract: Deep learning models have showcased remarkable performances in natural language processing tasks. While much attention has been paid to improvements in utility, privacy leakage and social bias are two major concerns arising in trained models. In this paper, we address both privacy protection and gender bias mitigation in classification models simultaneously. We first introduce a selective privacy-preserving method that obscures individuals’ sensitive information by adding noise to word embeddings. Then, we propose a negative multi-task learning framework to mitigate gender bias, which involves a main task and a gender prediction task. The main task employs a positive loss constraint for utility assurance, while the gender prediction task utilizes a negative loss constraint to remove gender-specific features. We have analyzed four existing word embeddings and evaluated them for sentiment analysis and medical text classification tasks within the proposed negative multi-task learning framework. For instances, RoBERTa achieves the best performance with an average accuracy of 95% for both negative and positive sentiment, with 1.1 disparity score and 1.6 disparity score respectively, and GloVe achieves the best average accuracy of 96.42% with a 0.28 disparity score for the medical task. Our experimental results indicate that our negative multi-task learning framework can effectively mitigate gender bias while maintaining model utility for both sentiment analysis and medical text classification.