Graph-Based Cooperation Multiagent Reinforcement Learning for Intelligent Traffic Signal Control

Jing Shang, Shunmei Meng, Jun Hou, Xiaoran Zhao, Xiaokang Zhou, Rong Jiang, Lianyong Qi, Qianmu Li

Abstract: In the trend of continuously advancing urban intelligent transport construction, traditional traffic signal control (TSC) struggles to make effective decisions with complex traffic conditions. Although multiagent deep reinforcement learning shows promise in optimizing traffic flow, most existing studies ignore the complex relationships between signal lights and fail to communicate with neighbors effectively. Moreover, the deterministic strategies generated by Q-learning-based methods struggle to be extended to large-scale urban road networks. Therefore, this article proposes a multiagent graph-based soft actor-critic (MAGSAC) approach for TSC, which combines graph neural networks with the soft actor-critic (SAC) algorithm and extends it to multiagent environments to address the TSC problem. Specifically, we employ graph-based networks and attention mechanism to expand the receptive domain of agents, enable environmental information to be shared among agents, and utilize the attention mechanism to filter out unimportant information. The algorithm adheres to the centralized training decentralized execution (CTDE) paradigm to minimize the nonstationarity of MARL. Finally, a rigorous experimental evaluation was conducted using the CityFlow simulator on both synthetic traffic grids and real-world urban road networks. Experimental results show that MAGSAC outperforms other TSC methods in performance metrics, including average queue length and waiting time, and achieves excellent performance under complex urban traffic conditions.

External IDs:dblp:journals/iotj/ShangMHZZJQL25