Personalized federated learning based on feature fusion

Wolong Xing, Zhenkui Shi, Hongyan Peng, Xiantao Hu, Yaozong Zheng, Xianxian Li

Abstract: Federated learning (FL) enables distributed clients to collaborate on training while storing their data locally to protect client privacy. However, due to data heterogeneity, including issues related to label distributions skew in heterogeneous scenarios, the resulting global model may not be suitable for all clients. In this work, we introduce a personalized federated learning method called pFedPM, which focuses on addressing this challenge of label distributions skew in heterogeneous scenarios. We replace traditional gradient uploading with feature uploading, and introduce a novel feature fusion scheme to learn personalized local model for clients. Specifically, the server receives feature information from clients, aggregates global features, and sends them back to the clients. Clients achieve personalization by fusing local and global features. Furthermore, we introduce a relation network as an additional decision layer, providing a non-linear learnable classifier to predict labels. Through the novel modeling techniques, our proposed method reduces communication costs and supports heterogeneous client models. Experimental results demonstrate that our approach outperforms recent FL methods on the MNIST, FEMNIST, and CIFAR-10 datasets while requiring less communication.