Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation

David Bruggemann, Christos Sakaridis, Tim Broedermann, Luc Van Gool

Abstract:

Standard unsupervised domain adaptation methods
adapt models from a source to a target domain using labeled
source data and unlabeled target data jointly. In
model adaptation, on the other hand, access to the labeled
source data is prohibited, i.e., only the source-trained
model and unlabeled target data are available. We investigate
normal-to-adverse condition model adaptation
for semantic segmentation, whereby image-level correspondences
are available in the target domain. The target
set consists of unlabeled pairs of adverse- and normal-condition
street images taken at GNSS-matched locations.
Our method—CMA—leverages such image pairs to learn
condition-invariant features via contrastive learning. In
particular, CMA encourages features in the embedding
space to be grouped according to their condition-invariant
semantic content and not according to the condition under
which respective inputs are captured. To obtain accurate
cross-domain semantic correspondences, we warp the normal
image to the viewpoint of the adverse image and leverage
warp-confidence scores to create robust, aggregated
features. With this approach, we achieve state-of-the-art semantic
segmentation performance for model adaptation on
several normal-to-adverse adaptation benchmarks, such as
ACDC and Dark Zurich. We also evaluate CMA on a newly
procured adverse-condition generalization benchmark and
report favorable results compared to standard unsupervised
domain adaptation methods, despite the comparative handicap
of CMA due to source data inaccessibility. Code is
available at https://github.com/brdav/cma.