Accelerating Fair Federated Learning: Adaptive Federated Adam
Abstract: Federated learning is a distributed and privacy-preserving approach to train a statistical model collaboratively from decentralized data of different parties. However, when datasets of participants are not independent and identically distributed, models trained by naive federated algorithms may be biased towards certain participants, and model performance across participants is non-uniform. This is known as the fairness problem in federated learning. In this paper, we formulate fairness-controlled federated learning as a dynamical multi-objective optimization problem to ensure fair performance across all participants. To solve the problem efficiently, we study the convergence and bias of Adam as the server optimizer in federated learning, and propose Adaptive Federated Adam (AdaFedAdam) to accelerate fair federated learning with alleviated bias. We validated the effectiveness, Pareto optimality and robustness of AdaFedAdam with numerical experiments and show that AdaFedAdam outperforms existing algorithms, providing better convergence and fairness properties of the federated scheme.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: #### Clarification of contents:
- Emphasized the use of the term "fairness" in the introduction section to reduce any ambiguities.
- Added underlying intuition behind the algorithm in Section 5.1 to help readers enhance the understanding of adafedadam.
- Clarified the use of all notations in a more formal and consistent manner throughout the manuscript to enhance clarity and readability.
#### Experimental results:
- Added a new baseline algorithm, FedProx, to compare with the proposed adafedadam.
- Updated the fine-tuning method for other algorithms and incorporated the revised experimental results to ensure fair comparison.
- Compared q-FedAvg with additional values of q, specifically q ∈ {1, 2, 4}, against AdaFedAdam, and presented the results in Figure 3 for a more comprehensive analysis.
#### Typos:
- Conducted a thorough examination of the manuscript and rectified all identified typos to enhance the overall clarity and coherence of the paper.
Assigned Action Editor: ~Virginia_Smith1
Submission Number: 1146
Loading