Counterfactual Fairness by Combining Factual and Counterfactual Predictions

Zeyu Zhou, Tianci Liu, Ruqi Bai, Jing Gao, Murat Kocaoglu, David I. Inouye

Keywords: Counterfactual Fairness, Fairness, Trustworthy ML

Abstract:

In high-stakes domains such as healthcare and hiring, the role of machine learning (ML) in decision-making raises significant fairness concerns.
This work focuses on Counterfactual Fairness (CF), which posits that an ML model's outcome on any individual should remain unchanged if they had belonged to a different demographic group.
Previous works have proposed methods that guarantee CF.
Notwithstanding, their effects on the model's predictive performance remain largely unclear.
To fill this gap, we provide a theoretical study on the inherent trade-off between CF and predictive performance in a model-agnostic manner.
We first propose a simple but effective method to cast an optimal but potentially unfair predictor into a fair one with a minimal loss of performance.
By analyzing the excess risk incurred by perfect CF, we quantify this inherent trade-off.
Further analysis on our method's performance with access to only incomplete causal knowledge is also conducted.
Built upon this, we propose a practical algorithm that can be applied in such scenarios.
Experiments on both synthetic and semi-synthetic datasets demonstrate the validity of our analysis and methods.

Primary Area: Fairness

Submission Number: 18604