CGNet: Classification-Guided Multi-Task Interactive Network for Hyperspectral and Multispectral Image Fusion

Guohua Lv, Yanlong Xu, Yongbiao Gao, Guixin Zhao, Guotao Wang, Xiangcheng Sun

Abstract: The goal of fusing hyperspectral images (HSI) and multispectral images (MSI) is to generate high-resolution hyperspectral images for downstream tasks. However, most existing methods overlook the specific requirements of these tasks, leading to a gap between the fusion process and its subsequent applications due to insufficient guidance from downstream tasks. To address this issue, we propose a classification-guided multitask interactive network (CGNet) that integrates both fusion and classification tasks into a unified framework, with two branches producing the fused image and classification results, respectively. In the fusion branch, we design a multi-level residual refinement module to efficiently integrate spatial and spectral information. Additionally, an attention-based multi-scale fusion module, incorporating both spatial and channel attention, is carefully crafted to enhance representation learning. In the classification branch, both 2-D and 3-D convolutions are employed to improve classification performance. Moreover, an information interaction module is proposed to guide the fusion task based on classification outcomes. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches on the Pavia Centre and Pavia University datasets.