MADA: Mask Aware Domain Adaptation for Open-set Semantic Segmentation

Keying Zhang

Keywords: Domain Adaptation, Open-set Semantic Segmentation

TL;DR: MADA investigates the similarities between visual and text modalities in both the source and target domains, aiming to align all modalities efficiently with few data and computational resources.

Abstract: Open-set semantic segmentation aims to identify categories that extend beyond the scope of the training data. Compared to the conventional semantic segmentation, open-set semantic segmentation constitutes a more practical and challenging scenario. Nonetheless, prevalent open-set semantic segmentation models predominantly incorporate extensive image-text datasets and substantial network architectures. Although the design enhances the comprehensive performance of these models, it also intensifies their computational demand, making them considerably challenging to train or fine-tune for adaptation to task-specific applications or domains. In this paper, we introduce a novel strategy called Mask Aware Domain Adaptation (MADA) for addressing open-set semantic segmentation challenges. MADA investigates the similarities between visual and text modalities in both the source and target domains, aiming to align all modalities efficiently with few data and computational resources. This alignment significantly enhances model performance in the target domain while simultaneously maintaining open-set capacity. Extensive experiments demonstrate the effectiveness and efficiency of our approach. We consider MADA to be a practical solution for scenarios which require high target domain performance as well as open-set flexibility capacity.

Submission Number: 14