Adversarial structured prediction for domain-adaptive semantic segmentation

Sudhir Yarram, Junsong Yuan, Ming Yang

Abstract: Semantic segmentation is a structured prediction problem that heavily relies on expensive annotated image data to train supervised models. Unsupervised domain adaptation has been successful in leveraging synthetic (source) images to build models that generalize well to real (target) image data without annotations. However, previous methods mainly utilize source ground truth for segmentation loss and do not fully utilize them for learning segmentation output structures to guide the target domain. In this work, we exploit similar output structures across domains in order to better segment the target images. Toward this end, we devise an adversarial structured prediction by utilizing a regularizer. This regularizer outputs structured predictions on provided image features. Using an adversarial training setup, we make the structured predictions follow the spatial layout learned from the source ground truth. As a result, even without an explicit alignment between source and target features, our proposed method can adapt well from a source to a target domain. We evaluate our method on different challenging synthetic-2-real benchmarks and validate the effectiveness of the proposed method when compared with the state of the arts.