A Feature Fusion-Based Transformer Network for Hyperspectral Super-Resolution

Minghua Wang, Bing Zhang, Jiaxin Li, Longfei Ren, Jocelyn Chanussot

Abstract: In recent years, massive efforts have been made to improve the spatial resolution of hyperspectral (HS) images with the assistance of other high-spatial-resolution (HR) imaging sources like RGB and multispectral sensors. Convolutional neural network (CNN)-based techniques are widely used to solve the HS image super-resolution (HISR) problem, but CNNs fail to capture long-range dependencies and global feature maps. In this paper, we design a novel feature fusion-based Transformer network (FF-Former) to capture modality-aware features by a feature-level fusion strategy. Multi-scale convolution feature extraction is employed to mine spatial information of HR images at different scales. A feature fusion-based approach is developed to enable adaptive fusion of HS and HR modalities, leveraging optimal fusion weights learned through training. Experimental results demonstrate the superiority of the FF-former in HISR tasks, achieving promising performance compared to existing fusion methods.