Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation

Roie Reshef, Kfir Yehuda Levy

TL;DR: An algorithm for private federated learning in the partial-participation setting with optimal excess loss and computational complexity.

Abstract:

This paper addresses the challenge of achieving Differential Privacy (DP) in Federated Learning (FL) under the partial-participation setting, where each machine participates in only some of training rounds.
While earlier work achieved optimal performance and efficiency in full-participation scenarios, these methods could not extend effectively to cases with partial-participation.
Our approach addresses this gap by introducing a novel noise-cancellation mechanism that ensures privacy without compromising convergence rates or computational efficiency.
We analyze our method within the Stochastic Convex Optimization (SCO) framework and demonstrate that it achieves optimal performance for both homogeneous and heterogeneous data distributions.
This work broadens the applicability of DP in FL, providing a practical and efficient solution for privacy-preserving learning in distributed systems with partial participation.

Lay Summary:

Machine Learning is a novel technique.
Machines can offer data to a server, and the server uses this data to learn how to solve a task for future data.
Ideally, the data given should not affect the result, but in practice, it does.
By looking at the result, you can infer what the data given by the machine was, which is a privacy breach.
To protect their data, the machine can distort it before sending it to the server, the more distortion we use, the more private the data become, but the less good the result is.
Our research is about creating an algorithm that can get more privacy with less distortion and proving its optimality and effectiveness.

Primary Area: Social Aspects->Privacy

Keywords: Machine Learning, Privacy, Federated Learning

Submission Number: 11657