Spurious Privacy Leakage in Neural Networks

TMLR Paper4898 Authors

Abstract:

Neural networks are vulnerable to privacy attacks aimed at stealing sensitive data.
The risks can be amplified in a real-world scenario, particularly when models are trained on limited and biased data.
In this work, we investigate the impact of spurious correlation bias on privacy vulnerability.
We introduce _spurious privacy leakage_, a phenomenon where spurious groups are significantly more vulnerable to privacy attacks than non-spurious groups.
We further show that group privacy disparity increases in tasks with simpler objectives (e.g. fewer classes) due to the persistence of spurious features.
Surprisingly, we find that reducing spurious correlation using spurious robust methods does not mitigate spurious privacy leakage.
This leads us to introduce a perspective on privacy disparity based on memorization, where mitigating spurious correlation does not mitigate the memorization of spurious data, and therefore, neither the privacy level.
Lastly, we compare the privacy of different model architectures trained with spurious data, demonstrating that, contrary to prior works, architectural choice can affect privacy outcomes.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Sanghyun_Hong1

Submission Number: 4898