FL Games: A federated learning framework for distribution shifts

TMLR Paper770 Authors

Abstract: Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server. However, participating clients typically each hold data from a different distribution, which can yield to catastrophic generalization on data from a different client, which represents a new domain. In this work, we argue that in order to generalize better across non-i.i.d. clients, it is imperative to only learn correlations that are stable and invariant across domains. We propose FL-Games, a game-theoretic framework for federated learning that learns causal features that are invariant across clients. While training to achieve the Nash equilibrium, the traditional best response strategy suffers from high-frequency oscillations. We demonstrate that FL-Games effectively resolves this challenge and exhibits smooth performance curves. Further, FL-Games scales well in the number of clients, requires significantly fewer communication rounds, and is agnostic to device heterogeneity. Through empirical evaluation, we demonstrate that FL Games achieves high out-of-distribution performance on various benchmarks.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

In this revision, we have endeavored to address the primary concerns raised by the reviewers while also conducting additional experiments to augment the support for our propositions in the paper. The main revisions are as follows:
- Theoretical proof for convergence of FL Games using Best Response Dynamics (BRD)
- Discussion on low training accuracy across benchmarks
- Discussion about the exact best response dynamics
- Additional baseline
- Correction of typographical errors and amelioration of manuscript writing

Assigned Action Editor: ~Yaoliang_Yu1

Submission Number: 770