Privacy-Preserving Data Filtering in Federated Learning Using Influence Approximation

Ljubomir Rokvic, Panayiotis Danassis, Boi Faltings

Keywords: privacy preserving, privacy, federated learning, vit, visual image transformers, image classification, transformers, filtering

TL;DR: Privacy preserving, influence based technique, to reliably filter out bad batches in Federated Learning.

Abstract: Federated Learning by nature is susceptible to low-quality, corrupted, or even malicious data that can severely degrade the quality of the learned model. Traditional techniques for data valuation cannot be applied as the data is never revealed. We present a novel technique for filtering, and scoring data based on a practical influence approximation (`lazy' influence) that can be implemented in a privacy-preserving manner. Each agent uses his own data to evaluate the influence of another agent's batch, and reports to the center an obfuscated score using differential privacy. Our technique allows for highly effective filtering of corrupted data in a variety of applications. Importantly, the accuracy does not degrade significantly, even under really strong privacy guarantees ($\varepsilon \leq 1$), especially under realistic percentages of mislabeled data.

Is Student: Yes