Group-Aware Threshold Adaptation for Fair Classification

Taeuk Jang, Pengyi Shi, Xiaoqian Wang

Keywords: Fairness, Post-Processing Method, Fairness-Accuracy Trade-Off

Abstract: The fairness in machine learning is getting increasing attention, as its applications in different fields continue to expand and diversify. To mitigate the discriminated model behaviors between different demographic groups, we introduce a novel post-processing method to optimize over multiple fairness constraints through group-aware threshold adaptation. We propose to learn adaptive classification thresholds for each demographic group by optimizing the confusion matrix estimated from the probability distribution of a classification model output. As we only need an estimated probability distribution of model output instead of the classification model structure, our post-processing model can be applied to a wide range of classification models and improve fairness in a model-agnostic manner to ensure privacy. This even allows us to post-process existing fairness methods to further improve the trade-off between accuracy and fairness. Moreover, our model is efficient with low computational cost by alternating optimization and flexible with the optimization over multiple fairness constraints. We provide Pareto frontier to characterize fairness-accuracy trade-off. Also, we provide a theoretical analysis of the optimal thresholds obtained from our model in terms of both accuracy and fairness in classification. Experimental results demonstrate that our method outperforms state-of-the-art methods and obtains the result that is closest to the theoretical accuracy-fairness trade-off boundary.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

TL;DR: We propose to learn adaptive classification thresholds for fair classification. Our method is computationally efficient and improves accuracy-fairness trade-off.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2111.04271/code)