FIFO: Learning Fog-invariant Features for Foggy Scene Segmentation

Sohyun Lee, Taeyoung Son, Suha Kwak

Keywords: Foggy scene understanding, Fog-invariant feature learning, Semantic segmentation

Abstract: Robust visual recognition under adverse weather conditions is of great importance in real-world applications. In this context, we propose a new method for learning semantic segmentation models robust against fog. Its key idea is to consider the fog condition of an image as its style and close the gap between images with different fog conditions in neural style spaces of a segmentation model. In particular, since the neural style of an image is in general affected by other factors as well as fog, we introduce a fog-pass filter module that learns to extract a fog-relevant factor from the style. Optimizing the fog-pass filter and the segmentation model alternately gradually closes the style gap between different fog conditions and allows to learn fog-invariant features in consequence. Our method substantially outperforms previous work on three real foggy image datasets. Moreover, it improves performance on both foggy and clear weather images, while existing methods often degrade performance on clear scenes.

One-sentence Summary: We present a new perspective on the effect of fog in semantic segmentation and propose FIFO, a new method based on fog-invariant feature learning for semantic foggy scene segmentation.