Cooperative Charging Stations Management Under Irrational Hierarchy EV Behaviors

Jie Liu, Shuoyao Wang, Xiaoying Tang

Abstract: The Internet of Things (IoT) technology connects various aspects of society and enhances human life. Electric vehicles (EVs) and charging stations (CSs) are essential components of the IoT system, offering business opportunities for companies like the aggregator. The aggregator can maximize profit by implementing various CSs management strategies, such as pricing and charging scheduling. However, managing CSs presents challenges due to irrational human behavior, particularly uncertain CS selections by EV users. To address this issue, we propose a cooperative model for CSs that incorporates the cognitive hierarchy quantum response CS selection model. In this model, EV users are considered to possess ${k}$ -level rationality, and the charging environment is formulated as a Markov decision process (MDP) based on this user model. To make optimal decisions for CSs from the MDP, we propose a denoising autoencoder-based deep reinforcement learning (DEEDRL) method. This method learns from the uncertain environment and effectively denoises the state space using a pretrained autoencoder. In addition, to reduce the computational burden caused by time-varying strategies, we design a discretization strategy for action space based on the current market rule of tiered pricing and CS types. Our experiments with real data demonstrate that our proposed method accurately portrays the CS selection behavior of irrational users in realistic scenarios. Furthermore, our method outperforms noncooperative modes and benchmark cooperative algorithms regarding profitability, such as DDPG and DQN.