Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

Filip Hanzely, Boxin Zhao, mladen kolar

Abstract: We investigate the optimization aspects of personalized Federated Learning (FL). We propose general optimizers that can be applied to numerous existing personalized FL objectives, specifically a tailored variant of Local SGD and variants of accelerated coordinate descent/accelerated SVRCD. By examining a general personalized objective capable of recovering many existing personalized FL objectives as special cases, we develop a comprehensive optimization theory applicable to a wide range of strongly convex personalized FL models in the literature. We showcase the practicality and/or optimality of our methods in terms of communication and local computation. Remarkably, our general optimization solvers and theory can recover the best-known communication and computation guarantees for addressing specific personalized FL objectives. Consequently, our proposed methods can serve as universal optimizers, rendering the design of task-specific optimizers unnecessary in many instances.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission: We revised the paper to incoporate the reviewers' advice. We also updated the reference. In addition, we improved the writing and formatting of the paper.

Code: https://github.com/boxinz17/PFL-Unified-Framework

Assigned Action Editor: ~Naman_Agarwal1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 559