Federated Learning Aggregation via a Reinforcement Learning Policy

Bear Häon, Adam Dunn, Jinman Kim, Chenyu Wang, Dongang Wang

Keywords: Federated Learning, Reinforcement Learning, Global Health

Abstract: Recent advances in federated learning have integrated an aggregation control policy trained with reinforcement learning. A research gap exists evaluating the performance impact of federated network elements as the reinforcement learning environment. This is particularly relevant for applications of machine learning in global health, which make use of federated learning to overcome cross-institution data-sharing constraints. In this work, we introduce a modular architecture of federated learning as a reinforcement learning environment. We conduct an experimental evaluation of policies trained in architecture configurations using a federated non-IID dataset and two deep reinforcement learning algorithms. Results of experiments show that choices of federated network elements only have a small effect on absolute classification accuracy (highest is 72.01%) for non-IID data, apart from the action aggregation strategy which is much lower. Findings are consistent with recent experiments, and this work provides a sandbox for robustly evaluating reinforcement learning methods in federated learning.