FedNHN: Generated Data-driven Personalized Federated Learning with Heterogeneous Data

Jialong Sun, Jing Dong, Zhanye Su, Wenchao Zhang, Kejia Zhang

Abstract: The emergence of heterogeneous data brings new challenges to Federated Learning (FL). Unlike homogeneous datasets, heterogeneous data are inherently characterized by sample domain center offsets, which makes traditional federated learning often suffer from poor model results due to catastrophic forgetting and increased communication costs when dealing with such data. To address this problem, we introduce FedNHN, a novel Personalized Federated Learning (PFL) algorithm. FedNHN regularizes the local training loss by quantifying the disparity between the local and global models to generate pseudo-data feature representations. Specifically, it generates two types of pseudo-data: Highly Pseudo and Perturbed Data, by employing the Fast Gradient Sign Method (FGSM) with both the global model and local data. Furthermore, FedNHN leverages the Hilbert-Schmidt Independence Criterion (DN-HSIC) to assess the distances between the feature representations of the generated data by the global model and the local model, respectively, and utilizes the Centered Kernel Alignment (CKA) similarity based on DN-HSIC to regulate the local training loss effectively. Our mathematical analyses show that our design can maintain the utility of the federated model under heterogeneous data, and extensive evaluations on three classical datasets illustrate the effectiveness and performance of our proposed implementation.

External IDs:dblp:conf/globecom/SunDSZ024