Semantic Affinity-Driven Spatiotemporal Transformer Network for Satellite Video Moving-Object Segmentation

Yixuan Lv, Zhuang Zhou, Kang Liu, Han Wang, Jian Yang, Shengyang Li, Jianing You, Kailun Zhang

Abstract: Satellite video intelligent processing plays a critical role in Earth observation applications such as traffic monitoring and environmental surveillance. However, moving-object segmentation in satellite videos faces several challenges. First, spatiotemporal redundancy makes it difficult to model long-range dependencies because large-scale scenes with slow background changes lead to fragmented segmentation. Second, semantic ambiguity arises when stationary objects like parked aircraft share category-level similarities with moving targets, which causes false positives. Besides, insufficient feature discrimination occurs as small, rigid objects such as ships exhibit weak texture and edge details under low-resolution imaging. To overcome these issues, we introduce a semantic affinity-driven spatiotemporal Transformer network that leverages a Transformer-based architecture to capture pixel-level dependencies across spatial and temporal dimensions. Furthermore, our network employs a contextual affinity-constrained decoder to suppress category-level interference and integrates a triple-branch feature extractor with edge priors for enhanced contour delineation. Our framework operates in an end-to-end manner without requiring fine-tuning during inference, which ensures deployment efficiency. Extensive experiments on a dataset built upon SAT-MTB demonstrate state-of-the-art performance with a J&F Mean of 71.7%. The proposed method outperforms the baseline by 3.7% with improvements of 4.4% in J-Mean and 3.1% in F-Mean. In addition, it surpasses the optimized SAM2 with a 10.7% higher J-Mean while maintaining a significantly smaller parameter count (34.6 M versus 224 M). Both qualitative and quantitative evaluations confirm the method’s superiority and temporal stability. This work offers a robust and efficient solution for accurate moving-object segmentation in satellite videos.

External IDs:dblp:journals/tgrs/LvZLWYLYZ25