Analyzing the Suitability of Cost Functions for Explaining and Imitating Human Driving Behavior based on Inverse Reinforcement Learning

Maximilian Naumann, Liting Sun, Wei Zhan, Masayoshi Tomizuka

Abstract: Autonomous vehicles are sharing the road with human drivers. In order to facilitate interactive driving and cooperative behavior in dense traffic, a thorough understanding and representation of other traffic participants' behavior are necessary. Cost functions (or reward functions) have been widely used to describe the behavior of human drivers since they can not only explicitly incorporate the rationality of human drivers and the theory of mind (TOM), but also share similarity with the motion planning problem of autonomous vehicles. Hence, more human-like driving behavior and comprehensible trajectories can be generated to enable safer interaction and cooperation. However, the selection of cost functions in different driving scenarios is not trivial, and there is no systematic summary and analysis for cost function selection and learning from a variety of driving scenarios. In this work, we aim to investigate to what extent cost functions are suitable for explaining and imitating human driving behavior. Further, we focus on how cost functions differ from each other in different driving scenarios. Towards this goal, we first comprehensively review existing cost function structures in literature. Based on that, we point out required conditions for demonstrations to be suitable for inverse reinforcement learning (IRL). Finally, we use IRL to explore suitable features and learn cost function weights from human driven trajectories in three different scenarios.