SCSG: Real-World Report Augmented Safety-Critical Scenario Generation for Autonomous Vehicles

Nigar Doga Karacik, Yingjie Xu, Xinyi Li, Yingbai Hu, Yinlong Liu

Keywords: Safety-Critical Scenario Generation, Autonomous Vehicles

Abstract: This study presents an end-to-end safety-critical scenario generation pipeline for autonomous driving vehicles, integrating data-driven, adversarial, and knowledge-based scenario generation techniques. The research utilizes 738 different crash reports, which encompass both traditional crash cases and incidents involving autonomous vehicles. The retrieval augmented generation (RAG) method is employed to convert these reports into structured scenario details, encompassing scenario descriptions, agent behaviors, road geometries, weather conditions, and agent spawn points relative to the ego vehicle. Once the scenario descriptions are established, our code generator module utilizes large language models (LLMs) to transform these structured details into Scenic code. To ensure the generated code functions correctly in the CARLA simulation environment, we have implemented an error correction module that verifies and maintains syntactic correctness. The error-free codes are selected for the risk assessment of the generated scenarios. For the risk assessment, 10 different metrics from SafeBench are used, and our scenarios are compared with the current baseline methods using three distinct ego vehicles pretrained by different reinforcement learning (RL) algorithms. Our generated scenarios demonstrated at least an 8\% improvement in the turning obstacle scenario category for all three pretrained reinforcement learning agents. For unprotected left turn, vehicle passing, red light running, and right turn scenarios, our method yielded the best results for at least one out of three RL ego agents. The results indicate that utilizing crash reports, increasing the number of adversarial agents in a scenario, and implementing various weather and road conditions enhance the complexity, variety, and real-world relevance of generated safety-critical scenarios.

Submission Number: 12