Preserving AUC Fairness in Learning with Noisy Protected Groups

Mingyang Wu, Li Lin, Wenbin Zhang, Xin Wang, Zhenhuan Yang, Shu Hu

TL;DR: We propose the first robust AUC fairness framework under noisy group labels with theoretical guarantees.

Abstract:

The Area Under the ROC Curve (AUC) is a key metric for classification, especially under class imbalance, with growing research focus on optimizing AUC over accuracy in applications like medical image analysis and deepfake detection. This leads to fairness in AUC optimization becoming crucial as biases can impact protected groups. While various fairness mitigation techniques exist, fairness considerations in AUC optimization remain in their early stages, with most research focusing on improving AUC fairness under the
assumption of clean protected groups. However, these studies often overlook the impact of noisy protected groups, leading to fairness violations in practice. To address this, we propose the first robust AUC fairness approach under noisy protected groups with fairness theoretical guarantees using distributionally robust optimization. Extensive experiments on tabular and image datasets show that our method outperforms state-of-the-art approaches in preserving AUC fairness. The code is in https://github.com/Purdue-M2/AUC_Fairness_with_Noisy_Groups.

Lay Summary:

In many AI applications, such as medical image analysis and deepfake detection, the Area Under the ROC Curve (AUC) is a crucial metric for evaluating how well a model performs. However, in situations where certain groups of people are underrepresented or have different characteristics, it's important to consider fairness alongside performance. Traditional fairness methods for AUC optimization often assume that groups are clearly defined and free from errors. In reality, these groups are often noisy, leading to unfair outcomes that can affect certain communities.

To solve this problem, we introduce a new approach to AUC fairness that works even when there is noise in the data regarding protected groups. By using a method called distributionally robust optimization, our approach ensures fairness while maintaining strong performance. Through rigorous testing on both tabular data and image data, we show that our method outperforms existing approaches in achieving fairness without sacrificing accuracy. This breakthrough helps create more reliable and unbiased AI systems for real-world applications.

Primary Area: General Machine Learning->Supervised Learning

Keywords: Robust Machine Learning, AUC Optimization, Fairness

Submission Number: 3239