Aggregation on Learnable Manifolds for Asynchronous Federated Optimisation

Archie Licudi, Anshul Thakur, Soheila Molaei, Danielle Belgrave, David A. Clifton

TL;DR: A new methodology for asynchronous federated learning over heterogenous clients inspired by polynomial mode connectivity and Riemannian optimisation, with experimental studies showing its promise.

Abstract: Asynchronous federated learning (FL) with heterogeneous clients faces two key issues: curvature-induced loss barriers encountered by standard linear parameter interpolation techniques (e.g. FedAvg) and interference from stale updates misaligned with the server’s current optimisation state. To alleviate these issues, we introduce a geometric framework that casts aggregation as curve learning in a Riemannian model space and decouples choice of update direction from staleness conflict resolution. Within this, we propose $\textbf{AsyncBezier}$, which replaces linear aggregation with low-degree polynomial (Bézier) trajectories to bypass loss barriers, and $\textbf{OrthoDC}$, which orthogonally projects delayed updates to reduce interference. We establish framework-level convergence guarantees covering each variant given simple assumptions on their components. On three datasets spanning general-purpose and healthcare domains, including LEAF Shakespeare and FEMNIST, our approach consistently improves accuracy and client fairness over strong asynchronous baselines; finally, we show that these gains are preserved even when other methods are allocated a higher local compute budget.

Submission Number: 2227