Knowledge-Assisted DRL for Energy Harvesting Based Multi-Access Wireless Communications

Sheng Luo, Haodi Zhang, Qifan Li, Kaishun Wu

Abstract: This paper investigates the use of deep reinforcement learning (DRL) to maximize the long-term throughput of multi-access wireless communication systems. Specifically, we consider the scenario in which an access point (AP) without dedicated power supply harvests energy from the ambient environment and uses the harvested energy to deliver data packets to multiple users. We formulate the access control problem as a Markov Decision Process (MDP) with unknown system dynamics and apply the deep reinforcement learning (DRL) approach to solve this problem, i.e., the optimal access strategy is approximated by using a double deep Q-network (DDQN). As the considered system usually has large state and action spaces, some domain knowledge is embedded into the training process to improve the performance of DDQN. It is shown that with the help of domain knowledge, access control strategies which can achieve a higher system throughput can be found by the knowledge-embedded DDQN. Moreover, the performance of the access control policy obtained by using the knowledge-embedded DDQN is compared with that of several conventional transmission schemes. Experiment results show that the transmission policy obtained by using our proposed model can achieve better performance.