Multi-representation decoupled joint network for semantic segmentation of remote sensing images

Xiaowei Lv, Rui Wang, Chengyu Zheng, Qicheng Yang, Zhaoxin Wang, Jie Nie

Abstract: In recent years, semantic segmentation has become an important means of processing remote sensing images, and it is widely used in various fields such as natural disaster detection, environmental protection, and land resource management. In response to this, the mainstream method of the deep convolutional network is constantly innovating and iterating. However, previous methods usually do not fully exploit the information associations between different representations, and the information of low-level representations is usually not well applied. In response to this, we propose a multi-representation decoupled joint network (MDJN) based on a three-branch architecture to improve the performance of semantic segmentation on remote sensing images, which utilizes multi-representation decoupling (MRD) to decouple the original single-branch network into the main branch, body branch and edge branch to enhance information fusion for different representations. Specifically, based on representation learning, we first propose a cross-representation graph convolution (CGC) module to fully mine and learn the context information between different representations with the help of graph convolutional networks (GCN). Secondly, we propose a new three-branch information interaction (TII) module to perform three-way interaction for the information of the three branches, so that the intra-class consistency and inter-class expressivity between different representations can fully play a role. The mean intersection over union (mIoU) of MDJN reaches 78.19% and 81.26% respectively on on both International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam datasets.