Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

Weijia ZHANG, Hao Liu, Fan Wang, Tong Xu, Haoran Xin, Dejing Dou, Hui Xiong

Abstract: Electric Vehicle (EV) has become a preferable choice in the modern transportation system due to its environmental and energy sus- tainability. However, in many large cities, EV drivers often fail to find the proper spots for charging, because of the limited charg- ing infrastructures and the spatiotemporally unbalanced charg- ing demands. Indeed, the recent emergence of deep reinforcement learning provides great potential to improve the charging expe- rience from various aspects over a long-term horizon. In this pa- per, we propose a framework, named Multi-Agent Spatio-Temporal Reinforcement Learning (Master), for intelligently recommending public accessible charging stations by jointly considering various long-term spatiotemporal factors. Specifically, by regarding each charging station as an individual agent, we formulate this prob- lem as a multi-objective multi-agent reinforcement learning task. We first develop a multi-agent actor-critic framework with the centralized attentive critic to coordinate the recommendation be- tween geo-distributed agents. Moreover, to quantify the influence of future potential charging competition, we introduce a delayed access strategy to exploit the knowledge of future charging compe- tition during training. After that, to effectively optimize multiple learning objectives, we extend the centralized attentive critic to multi-critics and develop a dynamic gradient re-weighting strategy to adaptively guide the optimization direction. Finally, extensive experiments on two real-world datasets demonstrate that Master achieves the best comprehensive performance compared with nine baseline approaches.