Towards Universal Debiasing for Language Models-based Tabular Data Generation

ACL ARR 2025 February Submission7313 Authors

Abstract: Large language models (LLMs) have excelled in various text generation tasks, including tabular data. However, inherent historical biases in tabular datasets often cause LLMs to propagate fairness issues, particularly when multiple advantaged and protected features are involved. In this work, we introduce a universal debiasing framework that minimizes dependencies at the group level by reducing the mutual information between advantaged and protected attributes simultaneously. By leveraging the autoregressive structure and analytic sampling distributions of LLM-based tabular data generators, our approach efficiently computes mutual information without resorting to cumbersome numerical estimations. Building on this foundation, we propose two complementary methods: a direct preference optimization (DPO)-based strategy, namely UDF-DPO, that integrates seamlessly with existing models, and a targeted debiasing technique, namely UDF-MIX, that achieves debiasing without tuning the parameters of LLMs. Extensive experiments demonstrate that our framework effectively balances fairness and utility, offering a scalable and practical solution for debiasing in high-stakes applications.

Paper Type: Long

Research Area: Ethics, Bias, and Fairness

Research Area Keywords: fairness; tabular data generation;

Languages Studied: English

Submission Number: 7313