VLEO Satellite Radio Access Network Slicing: A Hierarchical Deep Reinforcement Learning Approach

Juan Luo, Ruoyu Xiao, Peng Sun, Ying Qiao

Abstract: Network slicing is a key technology of Radio Access Networks (RAN), enabling support for heterogeneous service with various quality of service (QoS) requirements. However, effective RAN slicing solution remains a significant challenge in the time-varying satellite networks. Therefore, in this paper, a satellite RAN slicing framework with two time-scales and resource granularity is designed. In a large time-scale, radio resources are coarsely allocated to Very Low Earth Orbit (VLEO) satellites in the software-defined networking (SDN) level. In a small time-scale, radio resources and transmit power are finely allocated to ground users. An optimization problem is formulated to maximize ground users’ QoS satisfaction of slices and system Spectrum Efficiency (SE). So we propose a satellite network slicing algorithm based on a hierarchical deep reinforcement learning framework (SNS-HRL). In a large time-scale, the resource allocation problem in the SDN allocation level is modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). QMIX algorithm is utilized to allocate radio resources to different VLEO satellites. In a small time-scale, the resource allocation problem among slices is modeled as a Markov Decision Process (MDP), and the Proximal Policy optimization (PPO) algorithm is employed to schedule radio resources in the satellite allocation level. Additionally, resource allocation among users is achieved through a combination of the Proportional Fairness (PF) algorithm and the Lagrange multiplier method. Furthermore, simulation results demonstrate the effectiveness of the proposed satellite RAN slicing algorithm.