Federated Sampling with Langevin Algorithm under Isoperimetry
Abstract: Federated learning uses a set of techniques to efficiently distribute the training of a machine learning algorithm across several devices, who own the training data. These techniques critically rely on reducing the communication cost---the main bottleneck---between the devices and a central server. Federated learning algorithms usually take an optimization approach: they are algorithms for minimizing the training loss subject to communication (and other) constraints. In this work, we instead take a Bayesian approach for the training task, and propose a communication-efficient variant of the Langevin algorithm to sample \textit{a posteriori}. The latter approach is more robust and provides more knowledge of the \textit{a posteriori} distribution than its optimization counterpart. We analyze our algorithm without assuming that the target distribution is strongly log-concave. Instead, we assume the weaker log Sobolev inequality, which allows for nonconvexity.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: camera-ready version
Assigned Action Editor: ~Thang_D_Bui1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 1317
Loading