Distributional and Byzantine Robust Decentralized Federated Learning

Kaichuang Zhang, Ping Xu, Zhi Tian

Abstract: Decentralized federated learning (DFL) offers enhanced resilience to client failures and potential attacks than its centralized counterpart. This advantage stems from its ability to aggregate learning models from distributed clients requiring centralized server coordination. However, the practical adoption of DFL faces several challenges that threaten the robustness of local models. On one hand, the distribution of data might change over time, degrading the aggregated model’s performance on test data. On the other hand, Byzantine attacks, where certain users send malicious updates to their neighbors to spread erroneous knowledge, can compromise the convergence and accuracy of the global model. Notably, no existing work has simultaneously addressed both distributional shifts and Byzantine attacks in decentralized settings. To bridge this gap, we first propose a robust aggregation algorithm, Local Performance Evaluation with Temperature-Scaled Softmax Reweighting (LPE-TSR), to defend against Byzantine attacks. We then integrate Wasserstein distributionally robust optimization with LPE-TSR and develop Distributional and Byzantine Robust Decentralized Stochastic Gradient Descent (DB-Robust DSGD) to tackle both challenges simultaneously. DB-Robust DSGD allows flexible selection of robust aggregation algorithms tailored to specific scenarios. Experimental results show that LPE-TSR achieves optimal performance across diverse attack scenarios, while DB-Robust DSGD effectively mitigates both distributional shifts and Byzantine attacks.