Posterior Probability-Based Label Recovery Attack in Federated Learning

Rui Zhang, Song Guo

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Federated Learning, Label Leakage, Probability Estimation

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Recent works have proposed analytical attacks that can recover batch labels from gradients of a classification model in Federated Learning. However, these studies do not explain the essence of label leakage or show the scalability of other classification variants. In this paper, we demonstrate the root cause of label leakage from gradients and propose a generalized label recovery attack by estimating the posterior probabilities. Beginning with the focal loss function, we derive the relationship among the gradients, labels and posterior probabilities in a concise form. Then, we explain the essential reasons for such findings from the perspective of the exponential family. Furthermore, we empirically observe that positive (negative) samples of a class have approximate probability distributions. This key insight enables us to estimate the posterior probabilities of the target batch from an auxiliary dataset. Integrating the above elements, we finally present our label attack that can directly recover the batch labels of each class in realistic FL settings. Evaluation results show that on an untrained model, our attack can achieve over 96\% Class-level label Accuracy (ClsAcc) and 95\% Instance-level label Accuracy (InsAcc) on different groups of datasets, models and activations. For a training model, our approach reaches more than 90\% InsAcc on different batch sizes, class imbalance ratios, temperature parameters or label smoothing factors.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4626