Model-Based Robust Adaptive Semantic Segmentation

Jun Wang, Yiannis Kantaros

Keywords: Semantic Segmentation, Robustness, Natural Variation

Abstract: Semantic image segmentation enjoys a wide range of applications such as autonomous vehicles and medical imaging while it is typically accomplished by deep neural networks (DNNs). Nevertheless, DNNs are known to be fragile to input perturbations that are adversarially crafted or occur due to natural variations, such as changes in weather or lighting conditions. This issue of lack of robustness prevents the application of learning-based semantic segmentation methods on safety-critical applications. To mitigate this challenge, in this paper, we propose model-based robust adaptive training algorithm (MRTAdapt), a new training algorithm to enhance the robustness of DNN-based semantic segmentation methods against natural variations that leverages model-based robust training algorithms and generative adversarial networks. Natural variation effects are minimized from both image and label sides. We provide extensive experimental results on both real-world and synthetic datasets demonstrating that model-based robust adaptive training algorithm outperforms multiple state-of-the-art models under various natural variations.

One-sentence Summary: We propose a new method to enhance robustness of deep learning-based segmentation methods against natural variations, such as changes in the lighting conditions.