Self-Training and Curriculum Learning Guided Dynamic Refined Network for Remote Sensing Class-Incremental Semantic Segmentation

Hongbo Zhao, Ruimin Ren, Shuchang Lyu, Binghao Liu, Chunlei Wang, Meng Li, Guangliang Cheng, Qi Zhao

Abstract: Class-incremental semantic segmentation aims to update the segmentation model with training samples containing only novel categories. Within this domain, catastrophic forgetting is a common challenge. In remote sensing scenes, images always have large discrepancies caused by a large variety of geographical objects. Therefore, besides catastrophic forgetting, there exists two additional primary challenges persist. The first one is a huge imbalance in image categories while the second one is error accumulation during multiple incremental training steps. To solve these three problems, we propose a new Self-Training and Curriculum Learning Guided Dynamic Refined Network (STCL-DRNet). Specifically, we first design a self-training-based branch to ease the tendency of catastrophic forgetting. We then design a dynamic refined loss to mitigate the uneven category distribution. Through further embedding class-balanced curriculum learning, we can alleviate the performance drop from noisy accumulation. Extensive experiments on benchmark datasets, DeepGLobe and iSAID, prove that the proposed STCL-DRNet achieves new SOTA performance. Visualization and analysis further substantiate the interpretability.