FedGPA: Federated Learning with Global Personalized Aggregation

Zongfu Han, Yu Feng, Yifan Zhu, Zhen Tian, Fangyu Hao, Meina Song

Abstract: A significant challenge in Federated Learning (FL) is addressing the heterogeneity of local data distribution across clients. Personalized Federated Learning (PFL), an emerging method aimed at overcoming data heterogeneity, can either integrate personalized components into the global model or train multiple models to achieve personalization. However, little research has simultaneously considered both directions. One such approach involves adopting a weighted aggregation method to generate personalized models, where the weights are determined by solving an optimization problem among different clients. In brief, previous works either neglect the use of global information during local representation learning or simply treat the personalized model as learning a set of individual weights. In this work, we initially decouple the model into a feature extractor, associated with generalization, and a classifier, linked to personalization. Subsequently, we conduct local–global alignment based on prototypes to leverage global information for learning better representations. Moreover, we fully utilize these representations to calculate the distance between clients and develop individual aggregation strategies for feature extractors and classifiers, respectively. Finally, extensive experimental results on five benchmark datasets under three different heterogeneous data scenarios demonstrate the effectiveness of our proposed FedGPA.