Deep Reinforcement Learning for Intelligent Transportation Systems

Xiao-Yang Liu, Zihan Ding, Sem Borst, Anwar Walid

Abstract:

Intelligent Transportation Systems (ITSs) are envisioned to play a critical role in improving traffic flow and reducing congestion, which is a pervasive issue impacting urban areas around the globe. Rapidly advancing vehicular communication and edge cloud computation technologies provide key enablers for smart traffic management. However, operating viable real-time actuation mechanisms on a practically relevant scale involves formidable challenges, e.g., Markov Decision Processes (MDP) and conventional Reinforcement Learning (RL) techniques suffer from poor scalability
due to state space explosion. Motivated by these issues, we explore the potential for Deep Q-Networks
(DQN) to optimize traffic light control policies. As an initial benchmark, we establish that the DQN algorithms yield the ``thresholding" policy in a single-intersection.
Next, we examine the scalability properties of DQN algorithms and their performance in a linear network topology with several intersections along a main artery. We demonstrate that DQN algorithms produce intelligent behavior, such as the emergence of ``greenwave'' patterns, reflecting their ability to learn favorable traffic light actuations.

Keywords: Deep Q-learning, intelligent transportation system, thresholding policy, greenwave

TL;DR: We propose an adaptive deep reinforcement learning scheme for intelligent traffic light actuation, and observe that the `greenwave' phenomenon emerges without preprogramming the time offsets of traffic lights.