Testing Fairness with Utility Trade-offs: A Wasserstein Projection Approach

ICLR 2026 Conference Submission14786 Authors

Keywords: statistical test of fairness hypothesis, Wasserstein Projection, Optimal Transport, Strong Duality, Asymptotic Analysis

Abstract: Ensuring fairness in data-driven decision-making has become a central concern across domains such as marketing, lending, and healthcare, but fairness constraints often come at the cost of utility. We propose a statistical hypothesis testing framework that jointly evaluates approximate fairness and utility, relaxing strict fairness requirements while ensuring that overall utility remains above a specified threshold. Our framework builds on the strong demographic parity (SDP) criterion and incorporates a utility measure motivated by the potential outcomes framework in causal inference. The test statistic is constructed via Wasserstein projections, enabling auditors to assess whether observed fairness–utility trade-offs are intrinsic to the algorithm or attributable to randomness in the data. We show that the test is computationally tractable, interpretable, broadly applicable across machine learning models, and extendable to more general settings. We apply our approach to multiple real-world datasets, offering new insights into the fairness–utility trade-off through the perspective of statistical hypothesis testing.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 14786