ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression

Published: 19 Jun 2023, Last Modified: 21 Jul 2023FL-ICML 2023EveryoneRevisionsBibTeX
Keywords: Federated sampling, Langevin sampling, Error Feedback
TL;DR: In particular, we propose 3 federated sampling algorithms based on EF21, EF21-P and LMC, with uplink, donwlink and bidirectional compression.
Abstract: Federated sampling algorithms have recently gained great popularity in the community of machine learning and statistics. This paper studies variants of such algorithms called Error Feedback Langevin algorithms (ELF). In particular, we analyze the combinations of EF21 and EF21-P with the federated Langevin Monte-Carlo. We propose three algorithms: P-ELF, D-ELF, and B-ELF that use, respectively, primal, dual, and bidirectional compressors. We analyze the proposed methods under Log-Sobolev inequality and provide non-asymptotic convergence guarantees.
Submission Number: 42