Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees

Xin Yu, Zelin He, Ying Sun, Lingzhou Xue, Runze Li

Abstract: Personalized federated learning (PFL) offers a flexible framework for aggregating information across distributed clients with heterogeneous data. This work considers a personalized federated learning setting that simultaneously learns global and local models. While purely local training has no communication cost, collaborative learning among the clients can leverage shared knowledge to improve statistical accuracy, presenting an accuracy-communication trade-off in personalized federated learning. However, the theoretical analysis of how personalization quantitatively influences sample and algorithmic efficiency and their inherent trade-off is largely unexplored. This paper makes a contribution towards filling this gap, by providing a quantitative characterization of the personalization degree on the tradeoff. The results further offer theoretical insights for choosing the personalization degree. As a side contribution, we establish the minimax optimality in terms of statistical accuracy for a widely studied PFL formulation. The theoretical result is validated on both synthetic and real-world datasets and its generalizability is verified in a non-convex setting.

Lay Summary:

Personalized federated learning (PFL) has gained increasing attention due to the need for adapting global models to heterogeneous client data. However, achieving both high accuracy and communication efficiency and characterizing their trade-off under varying degrees of personalization remain largely unexplored in PFL.

In this work, we theoretically analyze the trade-off between statistical accuracy and communication cost under the influence of personalization degree. With a minimax theoretical guarantee, we show the benefit of PFL in generalization performance. These results offer practical guidance for designing communication-efficient personalized federated learning algorithms.

To the best of our knowledge, this is the first theoretical study to characterize the generalization performance of PFL while explicitly quantifying the trade-off between statistical accuracy and communication efficiency. Our analysis provides broad insights that inform the choice of personalization degree for a wide range of PFL algorithms to achieve high statistical accuracy and communication efficiency.

Link To Code: https://github.com/ZLHe0/fedclup

Primary Area: Theory->Learning Theory

Keywords: Personalized Federated Learning, Statistical Complexity, Communication Complexity

Submission Number: 14197