Object Segmentation Based on Pseudo Supervision Relearning Under Extremely Weak Annotations for Remote Sensing Images

Yanan Liu, Wanning Zhu, Libao Zhang

Abstract: Weakly annotated object segmentation for remote sensing images (RSIs) has attracted lots of attention due to its low labeling costs. However, annotating a huge amount of multiclass RSIs with image labels is still highly dependent on specific expert knowledge and, thus, requires considerable labeling costs. In this article, intending to further alleviate the labor-intensive labeling costs, we introduce a novel extremely weak annotation condition. In this condition, a substantial portion of samples are unlabeled, while merely a small number of samples are labeled with inexact image-level annotations. To achieve object segmentation under extremely weak annotations, we propose pseudo supervision relearning (PSRL), a novel three-stage framework with the core insight of effectively harnessing the potentially valuable supervision clues stored in abundant unlabeled data. In the first stage, the extremely weak annotations are switched to fine-grained but noisy pseudo supervision with the aid of image-level semantic learning and attention-guided data augmentation. Then, a category-aware dataset resplit strategy based on the masking perturbation mechanism is designed, aiming at adaptively selecting high-quality pixelwise pseudomasks from the artificially generated pseudo supervision and achieving class-balanced reliable-unreliable labels division. Ultimately, we devise a novel dynamic thresholding strategy (DTS)-guided relearning network to take full advantage of the valuable semantic information in the resplit dataset. Experimental results on two public RSI datasets show the effectiveness of the proposed framework. Utilizing less supervised information, the proposed method yields competing results compared to weakly supervised learning-based methods with complete image-level annotations.