Generation and Evaluation of Synthetic Data Containing Treatments

Harry Amad, Zhaozhi Qian, Dennis Frauen, Julianna Piskorz, Stefan Feuerriegel, Mihaela van der Schaar

Keywords: synthetic data, evaluation, generative models, metrics, treatment effect analysis

TL;DR: We identify problems with the standard synthetic data generation pipeline when producing data containing treatments, and we propose a set of metrics and a generation method which perform better in this setting.

Abstract: Causal inference on medical data, such as estimation of treatment effects, is crucial to ensure the efficacy and safety of medical interventions. However, privacy concerns frequently limit access to the patient data necessary for such analyses. Generative models can produce synthetic data that preserves privacy and closely approximates the real data distribution, yet existing methods are not designed for data containing treatments and the specific challenges their downstream use pose. With our work we establish a set of desiderata that synthetic data containing treatments should satisfy: preservation of (i) the covariate distribution, (ii) the treatment assignment mechanism, and (iii) the outcome generation mechanism. Based on these desiderata, we propose a principled set of evaluation metrics to assess such synthetic data. Finally, we present STEAM: a novel method for generating Synthetic data for Treatment Effect Analysis in Medicine. STEAM mimics the data-generating process of real-world data containing treatments, and can ensure differential privacy. We empirically demonstrate that STEAM achieves state-of-the-art performance across our metrics as compared to existing generative models, particularly as the complexity of the generative task increases.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3845