Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labels

Yuxin Tian, Mouxing Yang, Yuhao Zhou, Jian Wang, Qing Ye, Tongliang Liu, Gang Niu, Jiancheng Lv

TL;DR: This study reveals an phenomenon that the global model of FL exhibits slow memorization of noisy labels and propose FedGR to improve the label-noise robustness of FL based on this.

Abstract: Conventional federated learning (FL) heavily depends on high-quality labels, which are often impractical in the real world, leading to the federated label-noise (F-LN) problem. Worse still, the F-LN problem is exacerbated by the heterogeneity of FL, whereas clients experience different label-noise types, ratios, and data distribution. In this study, we first observe an intriguing phenomenon that the global model of FL exhibits a slow memorization of noisy labels, suggesting its ability to maintain reliable predictions and robust representations in FL. Motivated by this, we propose a novel method termed Federated Global Reviser (FedGR), a straightforward yet effective method comprising three modules that collaboratively rectify noisy labels and regularize local training. By exploiting this inherent property, FedGR improves the label-noise robustness of FL in a self-contained manner. Extensive experiments on three widely used F-LN benchmarks demonstrate the superior performance of FedGR, consistently outperforming eight state-of-the-art baselines even in severe label-noise and data heterogeneity. Code: https://github.com/cs-yuxintian/FedGR-ICML26

Lay Summary:

Training machine learning models often requires large amounts of correctly labeled data, but in real-world settings these labels can be wrong, incomplete, or inconsistent. This problem becomes even harder in federated learning, where many users or organizations train a shared model without sharing their private data, because each participant may have different kinds and amounts of labeling mistakes.

In this work, we found that the shared global model in federated learning does not quickly memorize incorrect labels. Instead, it tends to keep more reliable predictions and useful internal representations for longer than expected. Based on this insight, we developed FedGR, a method that uses the global model to help revise noisy labels and guide each participant’s local training.

FedGR works without requiring extra clean data or outside supervision. Across several challenging benchmarks, it performed better than existing methods, especially when labels were highly noisy and participants had very different data. This makes federated learning more reliable for real-world applications where perfect labels are difficult to obtain.

Link To Code: Code: https://github.com/cs-yuxintian/FedGR-ICML26

Primary Area: Deep Learning->Robustness

Keywords: Federated Learning, Learning with Noisy Labels

Submission Number: 14465