Uncovering Bias: Exploring Gender Dynamics in Distance-Aware Mixup Techniques

Samyak Jain, Parth Chhabra, Ramit Sawhney

Keywords: Bias in AI, Mixup, NLP, Social Computing

TL;DR: We present a comprehensive study of gender bias in distance aware mixup techniques through experiments with different mixup sampling strategies.

Abstract: Bias is a pervasive issue in machine learning and has implications in multiple AI applications, encompassing dimensions like gender, age, demographics, and social aspects. Complex models, including deep neural networks, transformers etc., often inherit biases and stereotypes during training, attributable to selection bias within training data and algorithmic creation processes. Augmentation techniques like Mixup exhibit promising potential as debiasing frameworks, leveraging specialized sampling strategies and spatial information for bias mitigation. In this study, we evaluate gender bias within the distance-aware mixing frameworks, while exploring diverse sampling strategies for mixup. Using the Trustpilot corpus, we conduct experiments quantitatively analyzing bias as error disparity, investigating the impact of distance thresholds and various gender-based criteria on mixup operations. Our quantitative analysis indicates that employing a cross-gender mixup strategy yields the most effective bias reduction. We also release the code for our work.

Submission Number: 238