Unsupervised Pretraining Framework Guided Hyperspectral and Multispectral Image Fusion

Ke Zheng, Aiyu Chen, Haoyang Yu, Jiaxin Li, Bing Zhang

Abstract: The fusion of hyperspectral images (HSIs) and multispectral images (MSIs) is crucial for overcoming the limitations of low spatial resolution in HSI. Currently, supervised learning methods tend to yield satisfactory integration results when applied to data distributions similar to those of the training set; however, they often exhibit insufficient generalization when confronted with real-world application scenarios. In contrast, unsupervised methods exhibit good generalization capabilities; however, they typically require careful tuning of hyperparameters to achieve satisfactory results, primarily due to the lack of sufficiently clear training objectives. To fully leverage the advantages of both supervised and unsupervised learning, this letter proposes an unsupervised pretraining framework (UPFW) guided fusion approach, which effectively enhances the performance of HSI-MSI fusion by introducing low-resolution supervised pretraining and full-resolution unsupervised adaptive strategy. Specifically, in the first stage, the model adapts to the learning spatial and spectral degradation parameter; in the second stage, we propose an adaptive fusion network (ADFNet) and conduct supervised learning on low-resolution scale to obtain a pretrained fusion network model with a clear objective-oriented; in the third stage, we utilize the pretrained model for full-resolution unsupervised fusion, thereby enhancing the model’s generalization capabilities and applicability. Experimental results show that compared to traditional methods and other deep learning approaches, the proposed method achieves significant advantages in spectral fidelity and spatial detail recovery across multiple public datasets.

External IDs:dblp:journals/lgrs/ZhengCYLZ25