A Learning-Based Stochastic Game for Energy Efficient Optimization of UAV Trajectory and Task Offloading in Space/Aerial Edge Computing

Jialiuyuan Li, You Shi, Chen Dai, Changyan Yi, Yuxiao Yang, Xiangping Zhai, Kun Zhu

Abstract: In this paper, we study the energy-efficient unmanned aerial vehicle (UAV) and low earth orbital (LEO) satellite assisted mobile edge computing (MEC) in space–air–ground integrated networks (SAGINs). The key challenge involves how to efficiently execute tasks offloaded from IoT devices, given limited on-board computation capabilities of UAVs and delay constraints of tasks. To address it, UAVs are first allowed to adjust their trajectories to accommodate as many tasks as possible from IoT devices. Subsequently, UAVs compute a portion of these tasks while delegating the rest to LEO satellites for further execution. Accordingly, we formulate a joint optimization problem including UAV trajectory planning, task offloading, and bandwidth allocations, aiming to maximize the long-term energy efficiency of UAVs and LEO satellites (i.e., the total data size in computation offloading divided by energy consumption of UAVs and LEO satellites). Considering the interactions among intelligent UAVs, this problem is reformulated as a set of interconnected multi-agent stochastic games, and the existence of corresponding Nash Equilibrium (NE) is theoretically proved. To obtain the NE, a multi-agent reinforcement learning-based algorithm, namely UTPTR, is developed concerning system dynamics and uncertainties. Simulations evaluate the performance of the UTPTR and demonstrate its superiority compared to existing approaches.