Feature-Based Test Scenario Selection in Automated Driving: Insights from SHAP Values

Tin Stribor Sohn, Nora Gorhan, Maximilian Dillitzer, Lukas Ewecker, Tim Brühl, Robin Schwager, Ann-Therese Nägele, Eric Sax

Abstract: Scenario-based testing is seen as a promising approach for evaluating automated driving systems, by grouping potential real-world events into representative test scenarios within their intended operational design domain. However, the complexity of the open world creates countless scenario variations, making it essential to select relevant scenarios efficiently while avoiding those that are not relevant for the test object. Traditional expert-driven methods struggle with this complexity, as causes for certain events related to the target domain are often unknown leading to incomplete specifications of the test space. This is highlighting the need for data-driven approaches to analyse and identify key parameters contributing to unwanted system behavior. In this paper a novel method for selecting scenario parameters is introduced, using explainable machine learning. It consists of two parts: a model trained on vehicle measurement data describing the operational design domain in order to predict test outcomes and a feature explainer based on SHapely Additive exPlanations to highlight important features contributing to these outcomes. The effectiveness of the proposed method is demonstrated using real driving data from a predictive Adaptive Cruise Control system. The results reveal the potential to improve testing efficiency and effectiveness in automated driving systems development through a test outcome prediction model and its feature explanations, reducing the reliance on domain-specific expertise for manual or rule-based test scenario selection, analysis and specification.

External IDs:dblp:conf/itsc/SohnGDEBSNS24