Multi-Patch Perception and Knowledge-Guided Network for Semantic Segmentation of Large-Scale Remote Sensing Image

Yijie Zhang, Jian Cheng, Ziying Xia, Wenqi Yin, Siyu Liu, Nyima Tashi

Abstract: Semantic segmentation of large-scale remote sensing (LSRS) images has always been a hot research topic. Recently, methods have been proposed that segment LSRS images by dividing them into small patches, which results in the neglect of inter-patch connections during the segmentation process, thereby causing incomplete global semantic information. To alleviate this problem, we propose a multi-patch perception and knowledge-guided netwok (MPKG-Net) for semantic segmentation of LSRS images. The MPKG-Net has two significant characteristics: (1) Multi-Patch Perception: Initially, MPKG-Net constructs multi-patch feature representation for patches relevant to the current segmented patch. Subsequently, the multi-patch feature is further processed through the multi-patch perception branch built upon the global feature selection block (GFSB). The extracted multi-patch features are incorporated into all encoder stages of MPKG-Net, enabling the model to perceive abundant global semantic information. (2) Knowledge-Guided: To effectively utilize multi-patch features and enhance local features, we employ a pre-trained base model on large-scale remote sensing image datasets to construct the knowledge-guided branch. This branch processes features from both the multi-patch perception branch and the local feature capture branch through the knowledge-guided multi-patch feature filtering module (KG-MFFM) and the knowledge-guided local feature enhancement module (KG-LFEM), respectively. Finally, all features are fused stage-by-stage within the multi-feature fusion branch. Our proposed model has achieved excellent performance on multiple publicly available LSRS images semantic segmentation datasets. The code will be available at https://github.com/zhangyijie199703/MPKG-Net.

External IDs:doi:10.1109/tgrs.2026.3656649