Understanding of Server-Assisted Federated Learning with Incomplete Client Participation

Haibo Yang, Peiwen Qiu, Prashant Khanduri, Minghong Fang, Jia Liu

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: federated learning, client participation, probably approximately correct, statistical learning

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Existing works in federated learning (FL) often assumes an ideal system with either full client or uniformly distributed client participation.
However, in practice, it has been observed that some clients may never participate in FL training (aka incomplete client participation) due to a myriad of system heterogeneity factors. To mitigate impacts of incomplete client participation, a popular approach is the server-assisted federated learning (SA-FL) framework, where the server is equipped with an auxiliary dataset. However, despite the fact that SA-FL has been empirically shown to be effective in addressing the incomplete client participation problem, there remains a lack of theoretical understanding for SA-FL. Meanwhile, the ramifications of incomplete client participation in conventional FL is also poorly understood. These theoretical gaps motivate us to rigorously investigate SA-FL. Toward this end, to fully understand the impact of incomplete client participation on conventional FL, we first show that conventional FL is {\em not} PAC-learnable under incomplete client participation in the worst case. Then, we show that the PAC-learnability of FL with incomplete client participation can indeed be revived by SA-FL, which theoretically justifies the use of SA-FL for the first time. Lastly, to provide practical guidance for SA-FL training under {\em incomplete client participation}, we propose the SAFARI (server-assisted federated averaging) algorithm that enjoys the same linear convergence speedup guarantees as classic FL with ideal client participation assumptions, offering the first SA-FL algorithm with convergence guarantee. Extensive experiments on different datasets show SAFARI significantly improve the performance under incomplete client participation.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8067