Deep Reinforcement Learning-based Computation Offloading in Vehicular Networks

Liwei Geng, Hongbo Zhao, Haoqiang Liu, Yujie Wang, Wenquan Feng, Lu Bai

Abstract: With the rapid development of 5G communications and the Internet of Things (IoT), vehicular networks have enriched people’s lives with abundant applications. Since most of such applications are computation-intensive and delay-sensitive, it is difficult to guarantee the requirements of low latency and low energy consumption by relying on vehicles only. In addition, low latency has posed great challenge to the cloud computing. Therefore, as a promising paradigm, Mobile Edge Computing (MEC) is developed for vehicular networks to relieve the pressure on vehicles, which means to offload tasks to edge servers. However, existing studies mainly consider a constant channel scenario and ignore load balancing of edge servers in the system. In this paper, deep reinforcement learning is adopted to build an intelligent offloading system, which can balance the load balancing in the time-varying channel scenario. First, we introduce a communication model and a calculation model. Then the offloading strategy is formulated as a joint optimization problem. Furthermore, a deep deterministic policy gradient (DDPG) algorithm based on priority experience replay in the distributed scheme, which considers the load balancing, is proposed. Finally, performance evaluations illustrate the effectiveness and superiority of the proposed algorithm.