Scaling Autonomous Driving Safety with Synthetic Data

Qitai Wang, Yuntao Chen, Lue Fan, Yingyan Li, Zhaoxiang Zhang, Tieniu Tan

Keywords: Driving Simulation, Data Synthesis, View Synthesis, End-to-End Driving, Corner Cases, Safe Driving

TL;DR: We propose a generative driving simulation pipeline that can scalably produce diverse driving corner cases, and we are the first to validate the benefit and scaling effect of synthetic training data on end-to-end autonomous driving.

Abstract:

Modern data-driven driving planners tend to perform suboptimally under safety-challenging cases that are underrepresented in training data.
Given the difficulty in collecting real-world data to cover all possible corner cases, scaling synthetic training data to enhance planning safety is of considerable value.
We propose SafeScale, a geometry-based generative driving simulation method that enables scalable generation of diverse driving corner cases.
We compose novel scenes by combining visual and behavioral assets from real-world data, enabling precise scenario customization and ensuring synthetic data diversity.
We employ a generative model to synthesize photorealistic camera observations along the simulated ego trajectory in novel scenes.
We analyze the types of corner cases that the state-of-the-art planner struggles to handle and use \methodname to synthesize corresponding scenarios as supplementary training data.
Experiments on the NAVSIM dataset demonstrate that scaling up the amount of synthetic training data continuously improves the planner’s performance on real-world data, exhibiting a clear data scaling effect.
With up to 100K additional synthesized training scenarios, the state-of-the-art end-to-end planner achieves a 28.6% reduction in collision failure cases, a 34.6% reduction in near-collision failure cases, and a 20.9% reduction in driveable area deviation failure cases on the NAVSIM test set.
Experiment results further show that synthetic data targeting each specific type of corner case yields highly selective improvements in planner performance under the responding scenario, and that the effects of synthetic data for different corner scenarios are independent and additive.
To our knowledge, this work presents the first effective demonstration of improving real-world driving performance via synthetic data.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 1245