Strategy-driven Bidirectional Efficient Adaptive Federated Learning

SLADS Section C Paper5 Authors

Abstract: We propose two new strategies within the framework of federated learning: NLA (New Lazy Aggregation) and AA (Accelerated Aggregation). The NLA strategy reduces the costs of communication and computation by adaptively skipping the gradient, and the AA ac- celerates the computation and reduces the communication cost by adaptively accruing the gradient computation. We propose six strategy-driven communication-efficient non-convex adaptive federated learning via NLA and AA. In particular, based on these novel strategies and compression techniques, we propose two new algorithms: FedBNLACA and FedBACA, to reduce bidirectional communication costs. We give a theoretical guarantee of these al- gorithms for client (full or partial) participation in correlation under a non-convex setting. In non-convex stochastic optimization full client participation setting, our proposed FedBN- LACA and FedBACA algorithms achieve the same convergence rate as its non-compact coun- terpart. We have demonstrated through extensive experiments that our protocol achieves efficient training in non-convex environments and is robust to large amounts of devices, partial participation, and unbalanced data.

Submission Type: Special issue on Statistics and AI

Submission Number: 5