Decentralized Two-Stage Federated Learning with Knowledge Transfer

Tong Jin, Siguang Chen

Abstract: Recently, federated learning has been widely applied, but the performance of the local model will be greatly decreased when the data distribution across different clients is heterogeneous. In this paper, we propose a decentralized two-stage federated learning scheme with knowledge transfer. Specifically, a ring federated learning model with knowledge transfer is constructed, by which users can acquire accurate service results. Meanwhile, a knowledge accumulation method based on conditional generative adversarial networks (CGAN) is designed, which can accumulate information from the previous edge nodes without compromising privacy. Furthermore, the accumulated global knowledge is learned by edge nodes through knowledge distillation to improve the performance of models on the global test dataset. Finally, the simulation results demonstrate that our proposed scheme achieves better classification accuracy and stability compared with the state-of-the-art methods.