MR.AsyncFL: Staleness-Aware Aggregation for Asynchronous Federated Learning via Model Replacement

Abstract: Asynchronous federated learning (AFL) has become increasingly popular and crucial for scalable, privacy-preserving machine learning across diverse and distributed edge devices. However, a fundamental challenge in FL is the staleness of client updates, which can degrade global model accuracy and slow down convergence as clients operate and communicate independently. Existing AFL methods typically address this issue by down-weighting stale updates, but outdated client information may still persist in the global model over time. In this paper, we propose MR.AsyncFL, a fully AFL framework based on model replacement. Upon receiving a new local model from a client, the server replaces that client’s previously cached contribution in the global model with the updated one, preserving the invariant that the global model remains a convex combination of the most recent available client models. To support this mechanism, we propose a recursive weight update scheme that preserves normalization in a lightweight and fully asynchronous manner. We further provide a convergence analysis for MR.AsyncFL under bounded staleness and client participation assumptions, and derive an $\mathcal{O}(T^{-1/4})$ convergence rate under a specific parameter scaling. Experiments on CIFAR-10 and CIFAR-100 under both IID and non-IID settings, with and without staleness thresholds, show that MR.AsyncFL consistently outperforms representative asynchronous baselines, such as FedAsync, TWAFL, and Rolling FedAvg, while maintaining strong robustness under severe staleness and system heterogeneity.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Antti_Koskela1

Submission Number: 8556