Learning Critically in Federated Learning with Noisy and Heterogeneous Clients

Chenrui Wu, Zexi Li, Fangxin Wang, Chao Wu

Keywords: Federated learning, Noisy labels, Class imbalance

Abstract: Federated learning (FL) is a distributed learning framework for collaboratively training models with privacy guarantee. Class imbalance problem is a main problem in FL with heterogeneous clients. Besides, Label noise is also an inherent problem in scenarios since clients have varied expertise in annotations. However, the co-existence of heterogeneous label noise and class-imbalance distribution in FL’s small local datasets renders conventional label-noise learning methods ineffective. Thus, in this paper, we propose algorithm FedCNI, including a noise-resilience local solver and a robust global aggregator, to address the challenges of noisy and highly-skewed data in FL without using an additional clean proxy dataset. For the local solver, we first design a prototypical classifier to detect the noisy samples by evaluating the similarity between samples and prototypes. Then, we introduce a curriculum pseudo labeling method with thresholds for different classes cautiously from the noisy samples. For the global aggregator, We aggregate critically by switching re-weighted aggregation from data-size to noise level in different learning periods. Experiments on real-world datasets demonstrate that our method can substantially outperform state-of-the-art solutions and is robust in mix-heterogeneous FL environments.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning