Model-Agnostic Round-Optimal Federated Learning via Knowledge Transfer

Qinbin Li, Bingsheng He, Dawn Song

Keywords: Federated Learning, Communication-Bounded Learning

Abstract: Federated learning enables multiple parties to collaboratively learn a model without exchanging their local data. Currently, federated averaging (FedAvg) is the most widely used federated learning algorithm. However, FedAvg or its variants have obvious shortcomings. It can only be used to learn differentiable models and needs many communication rounds to converge. In this paper, we propose a novel federated learning algorithm FedKT that needs only a single communication round (i.e., round-optimal). With applying the knowledge transfer approach, our algorithm can be applied to any classification model. Moreover, we develop the differentially private versions of FedKT and theoretically analyze the privacy loss. The experiments show that our method can achieve close or better accuracy compared with the other state-of-the-art federated learning algorithms.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: The paper presents a new federated learning framework with a single communication round.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2010.01017/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=Fu7XByivpx2