Fair Without Leveling Down: A New Intersectional Fairness Definition

Gaurav Maheshwari, Aurélien Bellet, Pascal Denis, Mikaela Keller

Submission Type: Regular Long Paper

Submission Track: Ethics in NLP

Submission Track 2: Machine Learning for NLP

Keywords: Fairness; Intersectional; Leveling Down

TL;DR: A new intersectional fairness definition which takes into account worst case performance.

Abstract:

In this work, we consider the problem of intersectional group fairness in the classification setting, where the objective is to learn discrimination-free models in the presence of several intersecting sensitive groups.
First, we illustrate various shortcomings of existing fairness measures commonly used to capture intersectional fairness.
Then, we propose a new definition called the $\alpha$-Intersectional Fairness, which combines the absolute and the relative performance across sensitive groups and can be seen as a generalization of the notion of differential fairness.
We highlight several desirable properties of the proposed definition and analyze its relation to other fairness measures.
Finally, we benchmark multiple popular in-processing fair machine learning approaches using our new fairness definition and show that they do not achieve any improvement over a simple baseline.
Our results reveal that the increase in fairness measured by previous definitions hides a ``leveling down'' effect, i.e., degrading the best performance over groups rather than improving the worst one.

Submission Number: 2067