FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Federated Learning, Semi-supervised Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We propose an innovative method, termed FedAnchor, which provides an innovative solution for federated semi-supervised learning.
Abstract: Federated learning (FL) is a distributed learning paradigm that allows devices to collaboratively train a shared global model while keeping the data locally. Due to the nature of FL, it provides access to an astonishing amount of training data for meaningful research and applications. However, the assumption that all of these private data samples include correct and complete annotations is not realistic for real-world applications. Federated Semi-Supervised Learning (FSSL) provides a powerful approach for training models on a large amount of data without requiring all data points to be completely labeled. In this paper, we propose FedAnchor, an innovative method that tackles the label-at-server FSSL scenario where the server maintains a limited amount of labeled data, while clients' private data remain unlabeled. FedAnchor introduces a unique double-head structure, with one anchor head attached with a newly designed label contrastive loss based on the cosine similarity to train on labeled anchor data to provide better pseudo-labels for faster convergence and higher performance. Following this approach, we alleviate the confirmation bias and over-fitting easy-to-learn data problems coming from pseudo-labeling based on high-confidence model prediction samples. We conduct extensive experiments on three different datasets and demonstrate our method can outperform the state-of-the-art method by a significant margin, both in terms of convergence rate and model accuracy.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3446
Loading