Consistency Learning based on Class-Aware Style Variation for Domain Generalizable Semantic Segmentation

Meng Yang

Abstract:

Domain generalizable (DG) semantic segmentation, i.e., a semantic segmentation model pretrained from a source domain performs
well in previously unseen target domains without any fine-tuning,
remains an open question. A promising solution is learning styleagnostic and domain-invariant features with stylized augmented
data. However, existing methods mainly focused on performing
stylization on coarse-grained image-level features, while ignoring
to explore fine-grained semantic style clues and high-order semantic context correlation, which are essential in enhancing the
generalization. Motivated by this, we propose a novel framework
termed Consistent Learning based on Class-Aware Style Variation
(CL-CASV) for DG semantic segmentation. Specifically, with the
guidance of class-level semantic information, our proposed ClassAware Style Variation (CASV) module simulates imaging object
and imaging condition style variation that can appear in complex
real-world scenarios, thus generating fine-grained class-aware
stylized images with rich style variation. Then the similarities between augmentations and original images are exploited via our
Self-Correlation Consistency Learning (SCCL) that mines global
context consistency from the views of channel correlation and
spatial correlation in the feature and prediction spaces. Extensive
experiments on mainstream benchmarks, including Cityscapes,
GTAV, BDD100K, SYNTHIA, and Mapillary, demonstrate the effectiveness of our method as it surpasses the state-of-the-art methods