TinyHazardSynth: Industrial Grade Realistic Data Augmentation for Autonomous Driving Using 3D Modeling and Depth-Aware Occlusion Model

YUFENG GAO

Keywords: Synthetic Data Generation, Autonomous Driving Model Auditing, Depth-Aware Occlusion Modeling, Robustness Testing

TL;DR: Industrial pipeline inserting photorealistic small hazards into dash-cams via NeRF + depth fusion, boosting detector accuracy (1.4pp↑) and recall (25%↑)

Abstract: TinyHazardSynth, a depth-aware synthetic model-auditing pipeline that stress-tests autonomous-driving detectors by controllably inserting photorealistic small road obstacles—situations rarely captured by public datasets—into real dash-cam videos. NeRF-reconstructed assets are rendered with exact camera intrinsics; precise occlusions arise from a two-stage fusion of sparse LiDAR and Depth-Anything depth that converts relative estimates to metric scale. A MaskFormer semantic prior prevents ground and hood clipping, and modular fog/shadow layers vary visibility to probe robustness. The fully-automated factory produces thousands of labelled frames per day and can wrap around any perception stack. Auditing an in-house detector uncovers a 25 % miss-rate on tiny hazards and, after targeted retraining on our clips, lifts accuracy by 1.4 pp, demonstrating the pipeline’s value for safety-critical model assessment. We leave as future work an investigation of how the same controllable-insertion pipeline adapts to other rare hazard types (e.g., deformable debris or transparent objects) and to public datasets such as nuScenes and KITTI.

Submission Number: 12