Tuning Fairness with Quantities that Matter

Thomas Kehrenberg, Zexun Chen, Novi Quadrianto

Abstract:

The issue of fairness in machine learning models has recently attracted a lot of attention as ensuring it will ensure continued confidence of the general public in the deployment of machine learning systems.
Here, we focus on mitigating the harm incurred by a biased system
that offers better outputs (e.g. loans, jobs) for certain groups than for others.
We show that bias in the output can naturally be handled in probabilistic models
by introducing a latent target output. % that will modulate the likelihood function.
This formulation has several advantages:
first, it is a unified framework for several notions of fairness such as Demographic Parity and Equalized Odds;
second, it is expressed as a marginalization instead of a constrained problem;
and third, it allows the encoding of our knowledge of what the bias in the outputs should be.
Practically, the second allows us to reuse off-the-shelf toolboxes, and the latter translates to the ability to control the level of fairness by
directly varying fairness target rates such as true positive rates and positive rates.
In contrast, existing approaches rely on intermediate, arguably unintuitive, control parameters such as covariance thresholds.