TSMCF: Transformer-Based SAR and Multispectral Cross-Attention Fusion for Cloud Removal

Hongming Zhu, Zeju Wang, Letong Han, Manxin Xu, Weiqi Li, Qin Liu, Sicong Liu, Bowen Du

Abstract: Optical remote sensing is heavily affected by clouds, and thus cloud removal is a critical task for obtaining accurate and reliable results. Traditional methods for remote sensing data reconstruction include spectral-based, spatial-based, time-based, and hybrid-based methods. However, these methods have inherent limitations in handling cloud removal due to the imaging mechanism of optical images. Synthetic Aperture Radar (SAR) images have been used in remote sensing image de-clouding in recent years due to their unique imaging mechanism. However, the domain gap between SAR images and multispectral images, the nonuniformity of cloud distribution, and the lack of mining self-similarity and correlation between different spectra of multispectral images pose challenges for SAR-based cloud removal. In this study, we propose a SAR-based cross-attention network for multispectral image de-clouding reconstruction. The cross-attention mechanism helps regions with heavy cloud cover to refer more to the features of SAR images during reconstruction while avoiding interference from the noise interference of SAR images. Experimental results show that the proposed method outperforms state-of-the-art methods in terms of reconstruction accuracy and visual quality, demonstrating its effectiveness and potential for cloud removal in remote sensing.

External IDs:doi:10.1109/jstars.2025.3538316