Learning from End User Data with Shuffled Differential Privacy over Kernel Densities

Tal Wagner

Keywords: differential privacy, shuffled differential privacy, kernel density estimation, kde

TL;DR: We present a method for collecting and learning a classifier from private data distributed across end users, via kernel density estimates in the shuffled DP model.

Abstract:

We study a setting of collecting and learning from private data distributed across end users.
In the shuffled model of differential privacy, the end users partially protect their data locally before sharing it, and their data is also anonymized during its collection to enhance privacy.
This model has recently become a prominent alternative to central DP, which requires full trust in a central data curator, and local DP, where fully local data protection takes a steep toll on downstream accuracy.

Our main technical result is a shuffled DP protocol for privately estimating the kernel density function of a distributed dataset, with accuracy essentially matching central DP.
We use it to privately learn a classifier from the end user data, by learning a private density function per class.
Moreover, we show that the density function itself can recover the semantic content of its class, despite having been learned in the absence of any unprotected data.
Our experiments show the favorable downstream performance of our approach, and highlight key downstream considerations and trade-offs in a practical ML deployment of shuffled DP.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7378