Accelerating Differentially Private Federated Learning via Adaptive Extrapolation

Shokichi Takakura, Seng Pei Liew, Satoshi Hasegawa

Keywords: federated learning, differential privacy

Abstract: DP-FedAvg is one of the most popular algorithms for federated learning (FL) with differential privacy (DP), but it is known to suffer from the slow convergence in the presence of heterogeneity among clients’ data. Most of the existing methods to accelerate DP-FL require 1) additional hyperparameters or 2) additional computational cost for clients. This is not desirable since 1) hyperparameter tuning is computationally expensive and data-dependent choice of hyperparameters raises the risk of privacy leakage, and 2) clients are often resource-constrained. To address this issue, we propose DP-FedEXP, which adaptively selects the global step size based on the diversity of the local updates without requiring any additional hyperparameters or client computational cost. We show that DP-FedEXP provably accelerates the convergence of DP-FedAvg and it empirically outperforms existing methods tailored for DP-FL.

Submission Number: 17