Reinforcement Learning for Falsification of Dynamic Driving Scenarios

Oliver Chang, Kay Vargas, Leilani H. Gilpin, Daniel J. Fremont

Keywords: reinforcement learning, falsification, simulation

TL;DR: Apply reinforcement learning to falsify autonomous driving systems.

Abstract: Falsification has been widely used to find failure cases for cyber-physical systems. In the domain of autonomous driving, falsification has recently been applied to find adversarial driving maneuvers which cause other vehicles to crash. However, these techniques have only been applied to simplistic scenarios and are limited in their ability to explore the high-dimensional space of possible maneuvers. Moreover, the quality and diversity of counter-examples discovered by these methods has not been systematically analyzed. In this work, we propose a reinforcement learning (RL)-based falsification framework that can discover complex adversarial maneuvers in diverse driving scenarios. We train a Soft Actor-Critic (SAC) agent to generate adversarial maneuvers, using the Scenic scenario description language to generate a wide range of training data. The victim vehicles are controlled by standard lane-changing and vehicle-following models [ 32 ] and simulated in MetaDrive. To support our approach, we optimize Scenic’s interface to RL environments to greatly improve throughput. Finally, we compare our approach to existing falsification methods, both in terms of their efficiency at finding counter-examples as well as the diversity and quality of their counter-examples. Our results suggest that RL-based falsification can be an effective tool for testing and validating autonomous vehicle systems.

Area: Engineering and Analysis of Multiagent Systems (EMAS)

Generative A I: I acknowledge that I have read and will follow this policy.

Submission Number: 1649