A deep reinforcement learning model for large-scale traffic signal control based on graph meta-learning using local subgraphs

Zhicheng Zhou, Ya Zhang, Xinde Li

Abstract: This paper investigates the problem of traffic signal control in large-scale road networks. A deep reinforcement learning model based on graph meta-learning using local subgraphs is proposed to control the traffic signal. The entire traffic network is represented as a graph by defining traffic lights as nodes and treating connections between intersections as edges. A graph neural network is used to enhance cooperation and communications between agents since information about neighbors is aggregated. To overcome the challenges in large-scale road networks, the proposed model employs a graph neural network on local subgraphs to reduce the difficulty of training in large-scale road networks. The model trained in small-scale traffic networks is transferred to a large-scale traffic network. Agent knowledge acquired from local subgraphs during the training of a small-scale road network confers advantages to the training of large-scale road networks under the resemblance between the structures of local subgraphs in small- and large-scale road networks. Furthermore, meta-learning is used to facilitate the model’s rapid adaptability to unseen large-scale road networks. The advantage of the double Q-learning network is taken to reduce overestimation. In experiments, real-world road networks and synthetic road networks comprising more than 1000 intersections are given to evaluate the effectiveness of the proposed model.