Causality for Tabular Data Synthesis: A High-Order Structure Causal Benchmark Framework

Zineb Senane, Axel Karlsson, Lele Cao, Oleg Smirnov, Cheng Zhang, Sahar Asadi, Hedvig Kjellstrom, Gustav Eje Henter, Ruibo Tu

Keywords: tabular data synthesis, benchmark framework, causal discovery, high-order structural information, causality, synthetic data evaluation

TL;DR: A benchmark framework leveraging causal graphs to systematically evaluate tabular synthesis models on capturing high-order causal information.

Abstract: Tabular synthesis models remain ineffective in capturing complex dependencies, and the quality of synthetic data is still insufficient for comprehensive downstream tasks, such as prediction under distribution shifts, automated decision-making, and understanding cross-table relationships. A major challenge present in tabular data is the lack of prior knowledge about high-order relationships, which is defined as multivariate structural causal dependencies beyond pairwise correlations. We argue that a systematic evaluation on high-order structural information is a crucial first step in addressing this issue in tabular data synthesis. In this paper, we present high-order structural causal information as a natural form of prior knowledge and introduce a benchmark framework to evaluate tabular synthesis models. This framework allows us to generate benchmark datasets through a flexible range of data generation processes, allowing for the training of tabular synthesis models using these datasets for further evaluation. We propose multiple benchmark tasks, high-order metrics, and causal inference tasks as downstream tasks for evaluating the quality of synthetic data generated by the trained models. Our experiments demonstrate the effectiveness of the benchmark framework in evaluating the model's ability to capture high-order structural causal information. Furthermore, our benchmarking results provide an initial assessment of state-of-the-art tabular synthesis models. These results reveal significant gaps between ideal and actual performance and highlight how baseline methods differ. We open source the benchmark framework, including both code and data along with documentation, to support further research and development in this area.

Primary Area: datasets and benchmarks

Submission Number: 19928