ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression

Avetik Karagulyan, Peter Richtárik

Keywords: Federated sampling, Langevin sampling, Error Feedback

TL;DR: In particular, we propose 3 federated sampling algorithms based on EF21, EF21-P and LMC, with uplink, donwlink and bidirectional compression.

Abstract:

Federated sampling algorithms have recently gained great popularity in the community of machine learning and statistics.
This paper studies variants of such algorithms called Error Feedback Langevin algorithms (ELF).
In particular, we analyze the combinations of EF21 and EF21-P with the federated Langevin Monte-Carlo.
We propose three algorithms: P-ELF, D-ELF, and B-ELF that use, respectively, primal, dual, and bidirectional compressors.
We analyze the proposed methods under Log-Sobolev inequality and provide non-asymptotic convergence guarantees.

Submission Number: 42