CADUI: Cross-Attention-Based Depth Unfolding Iteration Network for Pansharpening Remote Sensing Images

Zhixuan Li, Jinjiang Li, Fan Zhang, Linwei Fan

Abstract: Pansharpening is an important technology for remote sensing imaging systems to obtain high-resolution multispectral (HRMS) images. It mainly obtains HRMS images with uniform spectral distribution and rich spatial details by fusing low-resolution multispectral (LRMS) images and high spatial resolution panchromatic (PAN) images. Therefore, how to extract features completely and reconstruct images with high quality is critical to obtain ideal fusion images. In this article, we propose a new pansharpening method, called the Cross-Attention-based Depth Unfolding Iteration network for pansharpening remote sensing images (CADUI), which achieves the desired fusion effect by iteratively optimizing the deep prior regularization and combining it with a cross-attention mechanism. The network consists of two parts: optimized iterations of deep prior regularization (DEIN-Block) and cross-attention mechanism (CAFM-Block). Among them, DEIN-Block introduces the depth prior as an implicit regularization and improves the adaptability and representation ability of the relevant data of the reconstructed image through iteration. CAFM-Block realizes dual-branch fusion through cross-attention fusion and channel-attention fusion to achieve better fusion results. Simulation experiments and real experiments are carried out on the standard datasets: QuickBird (QB) and WorldView-2 (WV2). Through quantitative comparison and qualitative analysis, it is proven that the method is superior to the existing methods.