DP-REC: Private & Communication-Efficient Federated Learning

Aleksei Triastcyn, Matthias Reisser, Christos Louizos

Keywords: Federated learning, differential privacy, compression, communication efficiency

Abstract: Privacy and communication efficiency are important challenges in federated training of neural networks, and combining them is still an open problem. In this work, we develop a method that unifies highly compressed communication and differential privacy (DP). We introduce a compression technique based on Relative Entropy Coding (REC) to the federated setting. With a minor modification to REC, we obtain a provably differentially private learning algorithm, DP-REC, and show how to compute its privacy guarantees. Our experiments demonstrate that DP-REC drastically reduces communication costs while providing privacy guarantees comparable to the state-of-the-art.

One-sentence Summary: We combine differential privacy with recent advances in compression to jointly tackle communication efficiency and privacy in federated learning

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2111.05454/code)