When digital twin meets deep reinforcement learning in multi-UAV path planning

Siyuan Li, Xi Lin, Jun Wu, Ali Kashif Bashir, Raheel Nawaz

Abstract: Unmanned aerial vehicles (UAVs) path planning is one of the promising technologies in the fifth-generation wireless communications. The gap between simulation and reality limits the application of deep reinforcement learning (DRL) in UAV path planning. Therefore, we propose a digital twin-based deep reinforcement learning training framework. With the help of digital twin, DRL model can be trained more effectively deployed to real UAVs. In this training framework, we propose a deep deterministic policy gradient (DDPG) based multi-UAV path planning algorithm. Based on decomposed actor structure in DRL, we design a pooling-based combined LSTM network to better understand different state information in a multi-UAV path planning task. Moreover, we also establish a digital twin platform for multi-UAV system, which has a high degree of simulation and visualization. The simulation result shows that the proposed algorithm can achieve higher mean rewards, and outperforms DDPG in average arrival rate by more than 30%.