Personalized Federated Learning With Multiview Geometry Structure

Yihan Yan, Shen Wang, Fanghui Sun, Xiaojun Tong

Abstract: Federated learning (FL) is a distributed machine learning paradigm ensuring data privacy. However, the statistical heterogeneity poses a challenge to building a single model that can perform well across all clients’ data distributions. Personalized FL (PFL) has emerged as a solution to mitigate the impact of statistical heterogeneity by training separate models for various target data distributions. A crucial aspect of PFL is to utilize additional information in the federated system to assist in training personalized models. In this article, we introduce a novel PFL approach that leverages a multiview geometry structure (GPFL). GPFL formulates an optimization problem to determine the correlation weights among clients by utilizing the geometry structure composed of client gradient updates. It further builds information carriers that facilitate personalized training based on the weights. To accurately capture the correlations, we use both L2 distance and cosine similarity views to depict geometric similarity. In federated training, the discrepancy in timeliness and tendency to overfit to local data in gradient updates cause the geometric similarity of such updates to inadequately reflect the client relationships. Therefore, GPFL employs representative gradients extracted from the client’s historical gradients to infer the correlation weights. Experimental results on four data sets, convex and nonconvex objectives, and two FL settings demonstrate that our method outperforms several PFL methods.