SIL-LAND: Segmentation Incremental Learning in Aerial Imagery via LAbel Number Distribution Consistency

Junxi Li, Wenhui Diao, Xiaonan Lu, Peijin Wang, Yidan Zhang, Zhujun Yang, Guangluan Xu, Xian Sun

Abstract: Segmentation incremental learning (SIL) has received a lot of attention in recent years due to the ability to overcome the problem of catastrophic forgetting. Our study found that differences in label number distribution (LAND) affect the performance of SIL. Because the labels for pixels of the old category are marked as background when the model is trained on the new tasks, the LAND is inconsistent with static learning that is considered to be the upper bound on incremental learning, which hinders the mitigation of the catastrophic forgetting problem. In response to the above problems, we propose an incremental learning method named SIL-LAND, which improves the accuracy by making the LAND of our method close to that of static learning. From the perspective of high-level semantic labels, we propose the prototype update mechanism for the problem that nonadaptive representative prototypes ignore the sample diversity of semantic categories in remote sensing images. By compensating for the difference in LAND at the feature level, the distance between the prototype and the actual class center is reduced; aiming at the lack of semantic consistency between feature vectors and prototypes, we propose a similarity measure module to increase the intraclass similarity between the prototype and the corresponding feature vectors. From the perspective of one-hot labels, we propose label reconstruction, including foreground screening and background padding to make the number distribution of one-hot labels as close as possible to that of static learning. A series of experimental results demonstrates the effectiveness of our method.