Long-term Fairness with Selective Labels

ICLR 2026 Conference Submission21429 Authors

Keywords: long-term fairness, reinforcement learning, selective labels

TL;DR: We study satisfying long-term fairness with respect to labels in the selective labels setting

Abstract: Long-term fairness algorithms aim to satisfy fairness beyond static and short-term notions by accounting for the dynamics between decision-making policies and population behavior. Most previous approaches evaluate performance and fairness measures from observable features and a label, which is assumed to be fully observed. However, in scenarios such as hiring or lending, the labels (e.g., ability to repay the loan) are _selective labels_ as they are only revealed based on positive decisions (e.g., when loan is granted). In this paper, we study long-term fairness in the selective labels setting, and analytically show that naive solutions do not guarantee fairness. To address this gap, we then introduce a novel framework that leverages both the observed data and a label predictor model to estimate the true fairness measure value, by decomposing into the observed fairness and bias from labels predictions. This allows us to derive the sufficient conditions to satisfy true fairness from observable quantities by using the confidence on the predictor model. Finally, we rely on our theoretical results to propose a novel reinforcement learning algorithm for effective long-term fair decision-making with selective labels. In semisynthetic environments, the proposed algorithm reached comparable fairness and performance to an agent with oracle access to the true labels.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 21429