Efficient Weighting and Optimization in Federated Learning: A Primal-Dual Approach
Abstract: Federated learning has emerged as a promising approach for constructing a large-scale cooperative learning system involving multiple clients without sharing their raw data. However, the task of finding the optimal sampling weights for each client, given a specific global objective, remains largely unexplored. This challenge becomes particularly pronounced when clients' data distributions are non-i.i.d. (independent and identically distributed) and when clients only partially participate in the learning process.
In this paper, we tackle this issue by formulating the aforementioned task as a bi-level optimization problem that incorporates the correlations among different clients. To address this problem, we propose a double-loop primal-dual-based algorithm, designed specifically to solve the bi-level optimization problem efficiently. To establish the effectiveness of our algorithm, we provide rigorous convergence analysis under mild assumptions.
Furthermore, we conduct extensive empirical studies using both toy examples and learning models based on real datasets. Through these experiments, we verify and demonstrate the effectiveness of our proposed method.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Yiming_Ying1
Submission Number: 1358
Loading