Spurious Privacy Leakage in Neural Networks

Chenxiang Zhang, Jun Pang, Sjouke Mauw

Keywords: spurious correlation, membership inference, privacy, robustness, safety

Abstract:

Neural networks are vulnerable to privacy attacks aimed at stealing sensitive data. When trained on real-world datasets, these models can also inherit latent biases, which may further increase privacy risks. In this work, we investigate the impact of spurious correlation bias on privacy vulnerability, identifying several key challenges. We introduce spurious privacy leakage, a phenomenon where spurious groups can be up to 100 times more vulnerable to privacy attacks than non-spurious groups, and demonstrate how this leakage is connected to task complexity. Furthermore, while robust training methods can mitigate the performance disparity across groups, they fail to reduce privacy vulnerability, and even differential privacy is ineffective in protecting the most vulnerable spurious group in practice. Finally, we compare model architectures in terms of both performance and privacy, revisiting prior research with novel insights.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1112