Towards Out-of-federation Generalization in Federated Learning

Mengmeng Ma, Tang Li, Weisong Shi, Xi Peng

Keywords: Federated Learning; Data Heterogeneity; Robustness; Topology-aware

Abstract: Federated Learning (FL) is widely employed to tackle distributed healthcare data. Existing methods primarily focus on addressing in-federation data heterogeneity. However, we observe that they can suffer from significant performance degradation when applied to unseen clients for out-of-federation (OOF) generalization. The recent attempts to address generalization to unseen clients generally fail to scale up to large-scale distributed settings due to high communication overhead and convergence difficulty. And the communication efficient methods often yield poor OOF robustness. To achieve OOF-resiliency in a scalable manner, we propose Topology-aware Federated Learning (TFL) that leverages client topology - a graph representing client relationships - to effectively train robust models against OOF data. We formulate a novel optimization problem for TFL, consisting of two key modules: Client Topology Learning, which infers the client relationships in a privacy-preserving manner, and Learning on Client Topology, which leverages the learned topology to identify influential clients and harness this information into the FL optimization process to efficiently build robust models. Empirical evaluation on a variety of real-world datasets verifies TFL's superior OOF robustness and communication efficiency.

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4505