pFedLHNs: Personalized Federated Learning via Local Hypernetworks

Liping Yi, Xiaorong Shi, Nan Wang, Ziyue Xu, Gang Wang, Xiaoguang Liu

Abstract: As an emerging paradigm, federated learning (FL) trains a shared global model by multi-party collaboration without leaking privacy since no private data transmission between the server and clients. However, it still faces two challenges: statistical heterogeneity and communication efficiency. To tackle them simultaneously, we propose pFedLHNs, which assigns each client with both a small hypernetwork (HN) and a large target network (NN) whose parameters are generated by the hypernetwork. Each client pulls other clients’ hypernetworks from the server for local aggregation to personalize its local target model and only interacts the small hypernetwork with other clients via the central server to reduce communication costs. Besides, the server also aggregates received local hypernetworks to construct a global hypernetwork and uses it to initialize new joining out-of-distribution (OOD) clients for cold start. Extensive experiments on three datasets with Non-IID distributions demonstrate the superiority of pFedLHNs in the trade-off between model accuracy and communication efficiency. The case studies justify its tolerance to statistical heterogeneity and new OOD clients.