Out of the Box: Weather Augmentation for Enhanced Detection in Bad Visibility Conditions

Thomas Rothmeier, Mayara Dal Vesco Hoger, Bogdan Tomoyuki Nassu, Werner Huber, Alois C. Knoll

Abstract: Accurately detecting vehicles, pedestrians, and obstacles is crucial for the decision-making capabilities of autonomous vehicles. Although current methodologies demonstrate high detection accuracy under favorable environmental conditions, their performance significantly diminishes in bad visibility, such as fog, rain, or snow. This is due, at least in part, to the fact that these “edge-case“ scenarios are underrepresented in current datasets. This work introduces a novel approach that utilizes state-of-the-art diffusion models and Generative Adversarial Networks to artificially enhance clear weather images with simulated weather disturbances. In addition, we present a method for eliminating bounding boxes that become invalid due to severe weather conditions or image deformations caused by the augmentations. The effectiveness of our methods was assessed both qualitatively and quantitatively across a broad dataset under varied weather scenarios. The results demonstrate that our image augmentation techniques can enhance object detection performance, surpassing the clear weather baseline. Our approach for removing invalid bounding boxes consistently improves Average Precision by 2–3% across most tested augmentations and is most effective for improving the detection of small vehicles.