Training Fair Tabular Foundation Models

Patrik Kenfack, Jesse C. Cresswell, Anthony L. Caterini, Samira Ebrahimi Kahou, Ulrich Aïvodji

Keywords: Tabular Foundation Models, Fairness

TL;DR: This paper proposes FairTFM, a fairness-aware pretraining framework for tabular foundation models that can produce fairer predictions in a single forward pass, without task-specific retraining.

Abstract: Tabular Foundation Models (TFMs) have emerged as leading methods for tabular predictive tasks, leveraging in-context learning to predict on new data without task-specific training. Despite the increased use of TFMs in high-stakes decision-making, their fairness properties remain largely unexplored. In this work, we incorporate fairness constraints directly into TFM training, enabling fair predictions in a single forward pass. Our approach addresses two key challenges: limited access to sensitive attributes in training data, and the incompatibility of existing fairness techniques with the in-context learning paradigm. We propose FairTFM, a scalable training strategy based on synthetic fairness tasks and a fairness-aware architecture using a gradient reversal layer, which encourages the model to learn representations invariant to sensitive attributes. Experiments on 120 fairness tasks show consistent improvements in fairness while maintaining competitive accuracy.

Submission Number: 64