Decentralized Wireless Federated Learning With Differential Privacy

Shuzhen Chen, Dongxiao Yu, Yifei Zou, Jiguo Yu, Xiuzhen Cheng

Abstract: This article studies decentralized federated learning algorithms in wireless IoT networks. The traditional parameter server architecture for federated learning faces some problems such as low fault tolerance, large communication overhead and inaccessibility of private data. To solve these problems, we propose a decentralized wireless federated learning algorithm called DWFL. The algorithm works in a system where the workers are organized in a peer-to-peer and server-less manner, and the workers exchange their privacy preserving data with the analog transmission scheme over wireless channels in parallel. With rigorous analysis, we show that DWFL satisfies <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(\epsilon,\delta)$</tex-math></inline-formula> -differential privacy and the privacy budget per worker scales as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(\frac{1}{\sqrt{N}})$</tex-math></inline-formula> , in contrast with the constant budget in the orthogonal transmission approach. Furthermore, DWFL converges at the same rate of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(\sqrt{\frac{1}{TN}})$</tex-math></inline-formula> as the best known centralized algorithm with a central parameter server. Extensive experiments demonstrate that our algorithm DWFL also performs well in real settings.