Communication Efficient Federated Learning over Wireless Channels using Robust Count Sketches

TMLR Paper1150 Authors

Abstract:

Large-scale federated learning (FL) over wireless multiple access channels (MACs) has
emerged as a crucial learning paradigm with a wide range of applications. However, its
widespread adoption is hindered by several major challenges, including limited bandwidth
shared by many edge devices, noisy and erroneous wireless communications, and heterogeneous
datasets with different distributions across edge devices. To overcome these fundamental
challenges, we propose Federated Proximal Sketching (FPS), a novel federated learning
algorithm specifically designed for noisy and bandlimited wireless environments. FPS uses
a count sketch data structure to address the bandwidth bottleneck and enable efficient compression
while maintaining accurate estimation of significant coordinates. Moreover, FPS is
designed to explicitly address the bias induced by communications over noisy wireless channels.
We establish the convergence of the FPS algorithm under mild technical conditions. It
is worth noting that FPS is able to handle high levels of data heterogeneity across edge devices.
We complement the proposed theoretical framework with extensive experiments that
demonstrate the stability, accuracy, and efficiency of FPS in comparison to state-of-the-art
methods on both synthetic and real-world datasets. Overall, our results show that FPS is a
promising solution to tackling the above challenges of FL over wireless MACs.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

File seemed to be corrupted. Verified that this version opened in adobe acrobat and pdf viewer on the website.

Added a new lemma (Lemma 2 in the appendix) which quantifies the drift of iterates. Using this new lemma updated the proof of the main theorem to include the impact of local epochs $E$ in convergence.

Made some grammatical corrections.

Assigned Action Editor: ~Sebastian_U_Stich1

Submission Number: 1150