Federated Learning with Fair Averaging

Zheng Wang, Xiaoliang Fan, Jianzhong Qi, Chenglu Wen, Cheng Wang, Rongshan Yu

Abstract:

Fairness has emerged as a critical problem in feder-
ated learning (FL). In this work, we identify a cause
of unfairness in FL – conflicting gradients with
large differences in the magnitudes.
To address
this issue, we propose the federated fair averaging
(FedFV) algorithm to mitigate potential conflicts
among clients before averaging their gradients. We
first use the cosine similarity to detect gradient con-
flicts, and then iteratively eliminate such conflicts
by modifying both the direction and the magnitude
of the gradients. We further show the theoretical
foundation of FedFV to mitigate the issue conflict-
ing gradients and converge to Pareto stationary so-
lutions. Extensive experiments on a suite of fed-
erated datasets confirm that FedFV compares fa-
vorably against state-of-the-art methods in terms of
fairness, accuracy and efficiency. The source code
is available at https://github.com/WwZzz/easyFL.