A Mathematical Model and a Convergence Result for Totally Asynchronous Federated Learning

Didier El Baz, Jia Luo, Hao Mo, Lei Shi

Abstract: A totally asynchronous gradient algorithm with fixed step size is proposed for federated learning. A mathematical model is presented and a convergence result is established. The convergence result is based on the concept of macro iterations sequence. The interest of the contribution is to show that the asynchronous federated learning method converges when gradients of loss functions are updated by workers without order nor synchronization and with possible unbounded delays.