LanEvil: Benchmarking the Robustness of Lane Detection to Environmental Illusions

Tianyuan Zhang, Lu Wang, Hainan Li, Yisong Xiao, Siyuan Liang, Aishan Liu, Xianglong Liu, Dacheng Tao

Abstract: Lane detection (LD) is an essential component of autonomous driving systems, providing fundamental functionalities like adaptive cruise control and automated lane centering. Existing LD benchmarks primarily focus on evaluating common cases, neglecting the robustness of LD models against environmental illusions such as shadows and tire marks on the road. This research gap poses significant safety challenges since these illusions exist naturally in real-world traffic situations. For the first time, this paper studies the potential threats caused by these environmental illusions to LD and establishes the first comprehensive benchmark LanEvil for evaluating the robustness of LD against this natural corruption. We systematically design 14 prevalent yet critical types of environmental illusions (e.g., shadow, reflection) that cover a wide spectrum of real-world influencing factors in LD tasks. Based on real-world environments, we create 94 realistic and customizable 3D cases using the widely used CARLA simulator, resulting in a dataset comprising 90,292 sampled images. Through extensive experiments, we benchmark the robustness of popular LD methods using LanEvil, revealing substantial performance degradation (-5.37% Accuracy and -10.70% F1-Score on average), with shadow effects posing the greatest risk (-7.39% Accuracy). Additionally, we assess the performance of commercial auto-driving systems OpenPilot and Apollo through collaborative simulations, demonstrating that proposed environmental illusions can lead to incorrect decisions and potential traffic accidents. To defend against environmental illusions, we propose the Attention Area Mixing (AAM) approach using hard examples, which witness significant robustness improvement (+3.76%) under illumination effects. We hope our paper can contribute to advancing more robust auto-driving systems in the future. Part of our dataset and demos can be found at the anonymous website.

Primary Subject Area: [Content] Vision and Language

Secondary Subject Area: [Content] Vision and Language

Relevance To Conference: This research marks a pioneering step towards enhancing the robustness of Lane Detection (LD) systems in autonomous driving, a critical aspect of multimedia/multimodal processing. By introducing LanEvil, the first comprehensive benchmark specifically designed to evaluate LD models against environmental illusions—such as shadows and tire marks—our work directly addresses the overlooked but crucial challenge of ensuring safety and reliability in real-world traffic conditions. Our systematic identification and simulation of 14 types of environmental illusions across 94 realistic 3D scenarios result in a dataset of 90,292 images, offering an unprecedented resource for testing and improving LD methods. The experiments conducted demonstrate a significant vulnerability in current LD technologies, with notable performance degradations highlighted. Through the innovative Attention Area Mixing (AAM) technique, we not only diagnose these critical weaknesses but also propose a methodological advancement that significantly improves LD system resilience to such environmental factors. By doing so, this work contributes essential insights and tools to the field of multimedia/multimodal processing, pushing the boundaries of what is possible in autonomous driving technology and setting new benchmarks for future research and development.

Submission Number: 1807