Sdreplay: diffusion model for continual semantic segmentation in traffic scenarios

Jian Jiang, Yan Tian, Yongchuan Xu, Zhaocheng Xu, Xun Wang

Abstract: Traditional semantic segmentation conducts pixel-level classification on fixed classes, which results in catastrophic forgetting when fine-tuning the segmentation model on new data. Continual semantic segmentation has been introduced to address this challenge; however, replaying methods based on generative adversarial networks (GANs) cannot guarantee either semantic accuracy in generated images or distribution alignment between original training data and generated images. Motivated by the diffusion model, which inherently considers the entire data distribution, we propose a replay module named SDReplay with a dual-generator architecture to generate images of old classes with accurate semantics and an aligned distribution, where the Structure-Preserved Generator (SPG) synthesizes high-fidelity imagery with precise semantic consistency by leveraging structural priors, while the Distribution-Aligned Generator (DAG) ensures robust distributional fidelity for legacy classes through advanced token embedding optimization. The results in multiple datasets show that our approach improves the mean intersection-over-union (mIoU) by approximately 1.0%.

External IDs:dblp:journals/mms/JiangTXXW25