Multi-Scale Fusion Attention Network for Multispectral Worldview3 Data Road Segmentation

Zhonggui Tong, Yuxia Li, Jinglin Zhang, Yushu Gong, Lei He

Abstract: In recent years, many semantic segmentation methods based on convolutional neural networks (CNN) have been applied to road extraction, but objects with similar spectral characteristics to roads in RGB images and road occlusions cause the discontinuous output of road extraction. To ensure extraction performance, hyperspectral data is used as a supplement to RGB data to improve the ability of remote sensing image road extraction in this paper. This paper uses a multi-scale fusion attention network to combine RGB and multispectral imagery. First, band selection is used to select multispectral bands with high inter-class separability, and the Cross-Source Feature Recalibration Module (CSFR) is used to calibrate and fuse spectral features at different scales to achieve fusion of multi-source features at different scales. In addition, a multi-scale attention decoder is proposed to fuse multi-level road features and global context information. The proposed method was applied to the SpaceNet dataset and self-annotated images from Chongzhou, a representative city in China. Our method performs better over the baseline HRNet by a large margin of +6.38 IoU and +5.11 F1-score on the SpaceNet dataset, +3.61 IoU and +2.32 F1-score on the self-annotated dataset (ChongZhou dataset).