Long-Tailed Federated Learning with Fixed Classifier

Yi Li, Weichao Li, Xin Zheng, Haiyan Fu, Yanqing Guo

Abstract: Federated learning (FL) is a machine learning approach where multiple participants train a model together without sharing their privacy data. The challenge of long-tail data heterogeneity in FL causes imbalanced data distribution among clients and significant disparities in data quantities across different classes. To tackle the long-tail distribution in FL, in this paper, we draw inspiration from the equiangular tight frame to establish a fixed balanced classifier, enhancing the model’s generalization ability for classes with fewer instances. Therefore, feature extractors and a loss aligned with fixed classifiers are designed to address the long-tail distribution in FL. The algorithm also utilizes Sample Standardization and Batch Normalization to standardize the feature space of samples, preventing the impact of the magnitude difference of features across different classes on the prediction probability. In addition, we also propose a logit adjustment loss under a mixed low-temperature setting. Experiments demonstrate that our algorithm outperforms state-of-the-art methods in FL settings with long-tail distribution.

External IDs:dblp:conf/icmcs/LiLZFG25