Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data

Shlomi Hod, Lucas Rosenblatt, Julia Stoyanovich

Keywords: Differential Privacy, Synthetic Data, LLM, Classification, Pretraining, Hyperparameter Tuning

TL;DR: Do You Really Need Public Data? Public Data Surrogates for Differential Privacy on Tabular Data

Abstract:

Differentially private (DP) machine learning often relies on the availability of public data for tasks like privacy-utility trade-off estimation, hyperparameter tuning, and pretraining.
While public data assumptions may be reasonable in text and image data, they are less likely to hold for tabular data.
This work introduces the notion of "surrogate" public data — datasets generated independently of sensitive data, which consume no privacy budget and are constructed solely from publicly available metadata.
We automate the process of generating surrogate public data with large language models (LLMs); in particular, we propose two methods: direct record generation as CSV files, and automated structural causal model (SCM) construction for sampling records.
Through extensive experiments, we demonstrate that surrogate public tabular data can effectively replace traditional public data when pretraining differentially private tabular classifiers.
To a lesser extent, surrogate public data are also useful for hyperparameter tuning of DP synthetic data generators, and for estimating the privacy-utility tradeoff.

Submission Number: 72