Federated Learning under Evolving Distribution Shifts

TMLR Paper4539 Authors

Abstract: Federated learning (FL) is a distributed learning paradigm that facilitates training a global machine learning model without collecting the raw data from distributed clients. Recent advances in FL have addressed several considerations that are likely to transpire in realistic settings such as data distribution heterogeneity among clients. However, most of the existing works still consider clients' data distributions to be static or conforming to a simple dynamic, e.g., in participation rates of clients. In real FL applications, client data distributions change over time, and the dynamics, i.e., the evolving pattern, can be highly non-trivial. Further, evolution may take place from training to testing. In this paper, we address dynamics in client data distributions and aim to train FL systems from time-evolving clients that can generalize to future target data. Specifically, we propose two algorithms, FedEvolve and FedEvp, which are able to capture the evolving patterns of the clients during training and are test-robust under evolving distribution shifts. Through extensive experiments on both synthetic and real data, we show the proposed algorithms can significantly outperform the FL baselines across various network architectures.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=8T42p8eMDb

Changes Since Last Submission:

- We have revised our manuscript to clearly formulate the research problem and clarify our research specifically targets domain generalization rather than domain adaptation.
- We have added theoretical analysis to further justify our setting and the motivation behind our methods.
- We have included more comprehensive experiment discussions to provide deeper insights into the performance of our model.

Assigned Action Editor: ~Zachary_B._Charles1

Submission Number: 4539