Decentralized Federated Learning Under Communication Delays

Na Lee, Hangguan Shan, Meiyan Song, Yong Zhou, Zhongyuan Zhao, Xinyu Li, Zhaoyang Zhang

Abstract: Federated learning (FL) provides a privacy-preserving approach to training algorithms across multiple edge agents or servers without sharing raw data. Based on dynamic network topologies, we study a fully decentralized FL framework utilizing gradient descent with momentum (MGD) to accelerate the convergence rate. Due to bounded time-varying transmission delays, model updates are asynchronous between different agents, which leads to time-varying information update delay. Extensive experiments are carried out to demonstrate the performance of DMFL over some related algorithms and further analyze the influence of information update delay, network size, and the data distribution on the convergence performance of DMFL.