DECO-Bench: Unified Benchmark for Decoupled Task-Agnostic Synthetic Data Release

Farzaneh Askari, Lingjuan Lyu, Vivek Sharma

Keywords: Synthetic Data Generation, Decoupling Methods, Privacy-Utility Trade-offs, Privacy-Preserving Machine Learning, Stable Diffusion

TL;DR: Introducing a unified benchmarking framework for task-agnostic decoupling in privacy-preserving ML, filling a critical gap in the field.

Abstract:

In this work, we tackle the question of how to systematically benchmark task-agnostic decoupling methods for privacy-preserving machine learning (ML). Sharing datasets that include sensitive information often triggers privacy concerns, necessitating robust decoupling methods to separate sensitive and non-sensitive attributes. Despite the development of numerous decoupling techniques, a standard benchmark for systematically comparing these methods remains absent. Our framework integrates various decoupling techniques along with synthetic data
generation and evaluation protocols within a unified system. Using our framework, we benchmark various decoupling techniques and evaluate their privacy-utility trade-offs. Finally, we release our source code, pre-trained models, datasets of decoupled representations to foster research in this area.

Submission Number: 1809