Communication-Computation Efficient Secure Aggregation for Federated Learning

Beongjun Choi, Jy-yong Sohn, Dong-Jun Han, Jaekyun Moon

Keywords: Federated Learning, Privacy, Graphs, Secure Aggregation, Communication-Efficient, Computation-Efficient

Abstract: Federated learning has been spotlighted as a way to train neural network models using data distributed over multiple clients without a need to share private data. Unfortunately, however, it has been shown that data privacy could not be fully guaranteed as adversaries may be able to extract certain information on local data from the model parameters transmitted during federated learning. A recent solution based on the secure aggregation primitive enables privacy-preserving federated learning, but at the expense of significant extra communication/computational resources. In this paper, we propose communication-computation efficient secure aggregation which reduces the amount of communication/computational resources at least by a factor of $\sqrt{n/ \log n}$ relative to the existing secure solution without sacrificing data privacy, where $n$ is the number of clients. The key idea behind the suggested scheme is to design the topology of the secret-sharing nodes (denoted by the assignment graph $G$) as sparse random graphs instead of the complete graph corresponding to the existing solution. We first obtain a sufficient condition on $G$ to guarantee reliable and private federated learning. Afterwards, we suggest using the Erd\H{o}s-Rényi graph as $G$, and provide theoretical guarantees on the reliability/privacy of the proposed scheme. Through extensive real-world experiments, we demonstrate that our scheme, using only 50% of the resources required in the conventional scheme, maintains virtually the same levels of reliability and data privacy in practical federated learning systems.

One-sentence Summary: Inspired by graph theory, we suggest communication-computation efficient secure aggregation (CCESA) which maintains the privacy of federated learning by using significantly reduced communication/computational resources than the conventional wisdom.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=lfEVChMLFd