Open-Set Domain Adaptation for Semantic Segmentation

Jinwoo Choi, Seun-An Choe, Ahyung Shin, Keon-Hee Park, Gyeong-Moon Park

Abstract:

Unsupervised domain adaptation (UDA) for semantic
segmentation aims to transfer the pixel-wise knowledge
from the labeled source domain to the unlabeled target do-
main. However, current UDA methods typically assume
a shared label space between source and target, limiting
their applicability in real-world scenarios where novel cat-
egories may emerge in the target domain. In this paper, we
introduce Open-Set Domain Adaptation for Semantic Seg-
mentation (OSDA-SS) for the first time, where the target
domain includes unknown classes. We identify two major
problems in the OSDA-SS scenario as follows: 1) the exist-
ing UDA methods struggle to predict the exact boundary of
the unknown classes, and 2) they fail to accurately predict
the shape of the unknown classes. To address these issues,
we propose Boundary and Unknown Shape-Aware open-
set domain adaptation, coined BUS. Our BUS can accu-
rately discern the boundaries between known and unknown
classes in a contrastive manner using a novel dilation-
erosion-based contrastive loss. In addition, we propose
OpenReMix, a new domain mixing augmentation method
that guides our model to effectively learn domain and size-
invariant features for improving the shape detection of the
known and unknown classes. Through extensive experi-
ments, we demonstrate that our proposed BUS effectively
detects unknown classes in the challenging OSDA-SS sce-
nario compared to the previous methods by a large margin.
The code is available at https://github.com/KHU-
AGI/BUS.