Benchmarking Bird's Eye View Detection Robustness to Real-World CorruptionsDownload PDF

Published: 07 Apr 2023, Last Modified: 13 Apr 2023ICLR 2023 Workshop SR4AD HYBRIDReaders: Everyone
TL;DR: We benchmark and analyze the robustness of BEV detectors under real-world corruptions on our newly proposed nuScenes-C dataset. The benchmark is openly accessible at https://github.com/Daniel-xsy/RoboBEV.
Abstract: The recent advent of camera-based bird's eye view (BEV) detection algorithms exhibits great potential for in-vehicle 3D object detection. Despite the progressively achieved results on the standard benchmark, the robustness of BEV detectors has not been thoroughly examined, which is critical for safe operations. To fill in this gap, we introduce nuScenes-C, a test suite that encompasses eight distinct corruptions with a high likelihood to occur in real-world applications, including Bright, Dark, Fog, Snow, Motion Blur, Color Quant, Camera Crash, and Frame Lost. Based on nuScenes-C, we extensively evaluate a wide range of BEV detection models to understand their resilience and reliability. Our findings indicate a strong correlation between the absolute performance on in-distribution and out-of-distribution datasets. Nonetheless, there is considerable variation in relative performance across different approaches. Our experiments further demonstrate that pre-training and depth-free BEV transformation have the potential to enhance out-of-distribution robustness. The benchmark is openly accessible at https://github.com/Daniel-xsy/RoboBEV.
Track: Original Contribution
Type: Repository
0 Replies

Loading