Unsupervised Domain Adaptation for Semantic Segmentation using Depth Distribution

Quanliang Wu, Huajun Liu

Keywords: Unsupervised Domain Adaptation, Semantic Segmentation, depth density, multi-task learning, pseudo-labels refinement

Abstract: Recent years have witnessed significant advancements made in the field of unsupervised domain adaptation for semantic segmentation. Depth information has been proved to be effective in building a bridge between synthetic datasets and real-world datasets. However, the existing methods may not pay enough attention to depth distribution in different categories, which makes it possible to use them for further improvement. Besides the existing methods that only use depth regression as an auxiliary task, we propose to use depth distribution density to support semantic segmentation. Therefore, considering the relationship among depth distribution density, depth and semantic segmentation, we also put forward a branch balance loss for these three subtasks in multi-task learning schemes. In addition, we also propose a spatial aggregation priors of pixels in different categories, which is used to refine the pseudo-labels for self-training, thus further improving the performance of the prediction model. Experiments on SYNTHIA-to-Cityscapes and SYNTHIA-to-Mapillary benchmarks show the effectiveness of our proposed method.

TL;DR: A multi-task learning method in unsupervised domain adaptation for semantic segmentation using depth distribution.