Clustering-Guided Federated Learning of Representations
Keywords: Federated learning, self-supervised representation learning, clustering, KL divergence.
TL;DR: We propose a federated representation learning through clustering scheme (FedRLC) that aims to improve the performance of federated self-supervised learning.
Abstract: Federated self-supervised learning (FedSSL) methods have proven to be very useful in learning unlabeled data that is distributed to multiple clients, possibly heterogeneously. However, there is still a lot of room for improvement for FedSSL methods, especially for the case of highly heterogeneous data and a large number of classes. In this paper, we introduce federated representation learning through clustering (FedRLC) scheme that utilizes i) a crossed KL divergence loss with a data selection strategy during local training and ii) a dynamic upload on local cluster centers during communication updates. Experimental results show that FedRLC achieves state-of-the-art results on widely used benchmarks even with highly heterogeneous settings and datasets with a large number of classes such as CIFAR-100.
Submission Number: 24
Loading