DRL-Based Joint Resource Allocation and Device Orchestration for Hierarchical Federated Learning in NOMA-Enabled Industrial IoT

Tantan Zhao, Fan Li, Lijun He

Abstract: Federated learning (FL) provides a new paradigm for protecting data privacy in Industrial Internet of Things (IIoT). To reduce network burden and latency brought by FL with a parameter server at the cloud, hierarchical federated learning (HFL) with mobile edge computing (MEC) servers is proposed. However, HFL suffers from a bottleneck of communication and energy overhead before reaching satisfying model accuracy as IIoT devices dramatically increase. In this article, a deep reinforcement learning (DRL)-based joint resource allocation and IIoT device orchestration policy using nonorthogonal multiple access is proposed to achieve a more accurate model and reduce overhead for MEC-assisted HFL in IIoT. We formulate a multiobjective optimization problem to simultaneously minimize latency, energy consumption, and model accuracy under the constraints of computing capacity and transmission power of IIoT devices. To solve it, we propose a DRL algorithm based on deep deterministic policy gradient. Simulation results show proposed algorithm outperforms others.