KRADA: Known-region-aware Domain Alignment for Open-set Domain Adaptation in Semantic Segmentation

Chenhong Zhou, Feng Liu, Chen Gong, Rongfei Zeng, Tongliang Liu, William Cheung, Bo Han

Abstract: In semantic segmentation, we aim to train a pixel-level classifier to assign category labels to all pixels in an image, where labeled training images and unlabeled test images are from the same distribution and share the same label set. However, in an open world, the unlabeled test images probably contain unknown categories and have different distributions from the labeled images. Hence, in this paper, we consider a new, more realistic, and more challenging problem setting where the pixel-level classifier has to be trained with labeled images and unlabeled open-world images—we name it open world semantic segmentation (OSS). In OSS, the trained classifier is expected to identify unknown-class pixels and classify known-class pixels well. To solve OSS, we first investigate which distribution that unknown-class pixels obey. Then, motivated by the goodness-of-fit test, we use statistical measurements to show how a pixel fits the distribution of an unknown class and select highly-fitted pixels to form the unknown region in each test image. Eventually, we propose an end-to-end learning framework, known-region-aware domain alignment (KRADA), to distinguish unknown classes while aligning the distributions of known classes in labeled and unlabeled open-world images. The effectiveness of KRADA has been verified on two synthetic tasks and one COVID-19 segmentation task.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

All changes are highlighted
1. Modify the title.
2. Modify Section 2 (Problem Setup).
3. Add ablation studies.
4. Modify some descriptions about MSP in the draft.

Code: https://github.com/chenhong-zhou/KRADA

Assigned Action Editor: ~Brian_Kulis1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 654