DTEMPan: Dual Texture-Edge Maintaining Transformer for Pansharpening

Wuzhou Quan, Wen Guo

Abstract: Pansharpening plays a crucial role in the domain of remote sensing image processing, as it allows for the generation of high-resolution multispectral (HRMS) images. The main objective of pansharpening is to fuse low-resolution multispectral (LRMS) and high-resolution panchromatic (HRPAN) images, resulting in HRMS images that exhibit uniform spectra and enhanced spatial details. Consequently, the primary focus of related research is to preserve accurate features from both input images and achieve superior image reconstruction. In this article, we introduce a novel pansharpening framework called dual texture-edge maintaining transformer (DTEMPan). Our framework achieves exceptional fusion results by leveraging a novel, more interpretable, and powerful architecture that considers pansharpening as dual, distinct deep sub-semantic branches. It independently reconstructs sub-semantic layer information, leading to improved performance. The DTEMPan architecture incorporates a dual transformer design comprising shared perception encoders and two parallel, effective semantic-level decoders. The hybrid multiscale texture maintaining (HMSTM) decoder and the precise edge maintaining (PEM) decoder are responsible for reconstructing the general low-frequency and rare high-frequency signals, respectively. Through the integration of complementary information from both decoders, DTEMPan is capable of reconstructing accurate edges and high spatial information while preserving rich spectral details. Extensive experimental evaluations have demonstrated it significantly outperforms the state-of-the-art methods on a number of benchmarks. Our code is available at https://github.com/ D-Walter/DTEMPan.