Urban Driving with Multi-Objective Deep Reinforcement Learning

Changjian Li, Krzysztof Czarnecki

Abstract: Autonomous driving is a challenging domain that entails multiple aspects: a vehicle should be able to drive to its destination as fast as possible while avoiding collision, obeying traffic rules and ensuring the comfort of passengers. In this paper, we present a deep learning variant of thresholded lexicographic Q-learning for the task of urban driving. Our multi-objective DQN agent learns to drive on multi-lane roads and intersections, yielding and changing lanes according to traffic rules. We also propose an extension for factored Markov Decision Processes to the DQN architecture that provides auxiliary features for the Q function. This is shown to significantly improve data efficiency. \footnoteData efficiency as measured by the number of training steps required to achieve similar performance. We then show that the learned policy is able to zero-shot transfer to a ring road without sacrificing performance.