Delay-aware packet scheduling for massive MIMO beamforming transmission using large-scale reinforcement learning

Xinran Zhang, Songlin Sun

Abstract: This work addresses the massive multiple input multiple output (MIMO) beamforming scheduling problem. The scheduling model for packet transmission with paralleled caches using massive MIMO multicast beamforming is proposed by using reinforcement learning (RL) theory to denote the mapping function from the system state to the beamforming strategy. The delay-aware model-free RL scheduling problem for massive MIMO multicast beamforming is derived and analyzed in asymptotic condition. We show that the model-free RL method can be a complementary way to optimize the packet delay when the channel state information (CSI) is unavailable and the traditional convex optimization based methods are consequently ineffective. To derive low-complexity algorithm, we build the RL sub-problems for each specific multicast group which are asymptotically independent. The policy gradient method is used to solve the proposed RL problems. In numerical experiments we provide simulation results under different number of transmitting antennas and provide performance comparison between the proposed method and randomized method. It shows that even without using CSI, the RL based policy can still dynamically optimize the delay of the system and out-perform the randomized policy.