Objective-aware Traffic Simulation via Inverse Reinforcement Learning

Guanjie Zheng, Hanyang Liu, Kai Xu, Zhenhui Jessie Li

Abstract: Traffic simulators act as an essential component in the operating and planning of transportation sys- tems. Conventional traffic simulators usually em- ploy a calibrated physical car-following model to describe vehicles’ behaviors and their interactions with traffic environment. However, there is no uni- versal physical model that can accurately predict the pattern of vehicle’s behaviors in different sit- uations. A fixed physical model tends to be less effective in a complicated environment given the non-stationary nature of traffic dynamics. In this paper, we formulate traffic simulation as an in- verse reinforcement learning problem, and propose a parameter sharing adversarial inverse reinforce- ment learning model for dynamics-robust simula- tion learning. Our proposed model is able to imi- tate a vehicle’s trajectories in the real world while simultaneously recovering the reward function that reveals the vehicle’s true objective which is invari- ant to different dynamics. Extensive experiments on synthetic and real-world datasets show the su- perior performance of our approach compared to state-of-the-art methods and its robustness to vari- ant dynamics of traffic.