Rising from Ashes: Generalized Federated Learning via Dynamic Parameter Reset

Jiahao Wu, Ming Hu, Yanxin Yang, Xiaofei Xie, ZeKai Chen, Chenyu Song, Mingsong Chen

Keywords: Federated Learning

Abstract:

Although Federated Learning (FL) is promising in privacy-preserving collaborative model training, it faces low inference performance due to heterogeneous data among clients.
Due to heterogeneous data in each client, FL training easily learns the specific overfitting features.
Existing FL methods adopt the coarse-grained average aggregation strategy, which causes the global model to easily get stuck in local optima, resulting in low generalization of the global model.
Specifically, this paper presents a novel FL framework named FedPhoenix to address this issue, which stochastically resets partial parameters to destroy some features of the global model in each round to guide the FL training to learn multiple generalized features for inference rather than specific overfitting features.
Experimental results on various well-known datasets demonstrate that compared to SOTA FL methods, FedPhoenix can achieve up to 20.73\% accuracy improvement.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 6923