Exploring Fairness-Accuracy Trade-Offs in Binary Classification: A Comparative Analysis Using Modified Loss Functions

Christina Trotter, Yixin Chen

Abstract: In this paper, we explore the trade-off between fairness and accuracy when data is biased and unbiased. We introduce two versions of a modified loss function: Group Equity and Group Equality Binary Cross Entropy (BCE). Both functions add the maximum group loss to the standard BCE loss and use a hyperparameter to adjust its significance. Group Equity BCE is more suited for unbiased data, while Group Equality BCE is useful for heavily bias data. We test both by training logistic regression models on Home Mortgage Disclosure Act (HMDA) data. We evaluate our data and models using multiple fairness metrics, including p-value and statistical parity difference. We observe race and race-sex group bias. Our results show that Group Equity BCE maintains high accuracy while slightly improving group fairness. Group Equality BCE in- creases group fairness substantially. However, there is usually a steep decline in accuracy.