Balanced Learning for Domain Adaptive Semantic Segmentation

Wangkai Li, Rui Sun, Bohao Liao, Zhaoyang Li, Tianzhu Zhang

Abstract:

Unsupervised domain adaptation (UDA) for semantic segmentation aims to transfer knowledge from a labeled source domain to an unlabeled target domain.
Despite the effectiveness of self-training techniques in UDA, they struggle to learn each class in a balanced manner due to inherent class imbalance and distribution shift in both data and label space between domains.
To address this issue, we propose Balanced Learning for Domain Adaptation (BLDA), a novel approach to directly assess and alleviate class bias without requiring prior knowledge about the distribution shift.
First, we identify over-predicted and under-predicted classes by analyzing the distribution of predicted logits.
Subsequently, we introduce a post-hoc approach to align the logits distributions across different classes using shared anchor distributions.
To further consider the network's need to generate unbiased pseudo-labels during self-training, we estimate logits distributions online and incorporate logits correction terms into the loss function.
Moreover, we leverage the resulting cumulative density as domain-shared structural knowledge to connect the source and target domains.
Extensive experiments on two standard UDA semantic segmentation benchmarks demonstrate that BLDA consistently improves performance, especially for under-predicted classes, when integrated into various existing methods.

Lay Summary: AI models for image segmentation often perform poorly when applied to new environments, especially for classes that are harder to recognize, like bicycles or traffic signs. This happens because models tend to favor "easier" or more frequent classes, leading to unbalanced learning. We propose BLDA, a new method that detects and corrects this imbalance by analyzing the model’s own prediction patterns—without needing prior knowledge about the new domain. BLDA adjusts the learning process on the fly to ensure fair treatment of all classes. This leads to more balanced and accurate predictions, especially in challenging real-world scenarios like autonomous driving.

Primary Area: Applications->Computer Vision

Keywords: Semantic segmentation, Unsupervised domain adaptation

Submission Number: 6487