Byzantine-Robust Aggregation for Federated Learning with Reinforcement Learning

Sizheng Yan, Junping Du, Zhe Xue, Ang Li

Abstract: Federated learning (FL) is a promising distributed machine learning approach that aims to address the challenges of data isolation and data privacy protection, thereby fostering the collaborative training of models and the sharing of information among multiple parties. However, the distributed architecture of Federated Learning relies on aggregating parameter updates from multiple devices, posing potential security risks from malicious entities. In this paper, we propose a dynamic robustness aggregation strategy based on reinforcement learning (RL), called RL-GM. This approach employs geometric median as a robust aggregation method and integrates federated learning parameters and historical processes within a RL framework. We utilize the deep deterministic policy gradient (DDPG) network as the RL agent that dynamically adjusts geometric median weights to mitigate the influence of outliers. We conduct experiments on three datasets and compare our method with other robust aggregation approaches. The results indicate that our method maintains strong performance in scenarios with poisoned data models, and exhibit superior robustness and stability.