Using maximal information auxiliary variables to improve synthetic data generation based on TabPFN foundation models

ICLR 2026 Conference Submission14659 Authors

Keywords: tabular synthetic data generation, in-context learning, tabular foundation models

TL;DR: We introduce MIAV, a strategy for enhancing synthetic data generation with TabPFN and other PFN-based tabular models. MIAV is theoretically grounded, improves weakly-associated variable handling, boosts efficiency, is order-invariant, and competitive

Abstract: Synthetic data generation for tabular datasets is shifting toward the use of large, general-purpose foundation models. TabPFN, a state-of-the-art example, uses in-context learning to generate probabilistic predictions conditioned on observed examples in a single forward pass. However, when variables are only weakly associated with others, the model's ability to generate realistic synthetic data deteriorates, as the context examples provide little predictive signal. To address this, we introduce the maximal information auxiliary variable (MIAV) strategy, which increases context information with auxiliary variables constructed by rank-matching random noise variables to real data. We establish theoretical properties of the approach which explain its good performance for weakly associated variables. Additional practical advantages of the MIAV approach include improved computational efficiency and invariance to variable order during the synthetic data generation process. Empirical evaluations, on simulated and real datasets, illustrate how the MIAV strategy improves data generation when compared to direct application of TabPFN, and is competitive against other baselines. To illustrate the generality of the MIAV approach we also present an implementation based on the TabICL model (a more scalable tabular foundation model restricted to classification tasks) for performing synthetic data generation on categorical datasets. Overall, MIAV offers an effective foundation model–based alternative to bespoke synthetic data generators.

Primary Area: generative models

Submission Number: 14659