FBS-PS: Fully Band-Separable PAN-Sharpening Considering the Physical Characteristics of Electro-Optical Sensors

Hyun-Ho Kim, Munchurl Kim

Abstract: Electro-optical (EO) satellites are primarily used for reconnaissance, national defense, and cartography. However, most high-resolution (HR) EO satellites obtain images at a lower resolution (LR) in the multispectral (MS) band compared to the panchromatic (PAN) band due to technical limitations. Deep learning-based PAN-sharpening methods have been continuously developed to address the growing demand for MS images with the same ground sample distance (GSD) as PAN images. The improvements in deep learning-based PAN-sharpening methods have focused on enhancing the network structure and often overlooked the physical characteristics of satellite EO sensors, leading to artifacts such as noise and color distortions in the PAN-sharpened (PS) images. Thus, we propose a fully band-separable PAN-sharpening (FBS-PS) method and its more elaborate quality-centric model, called FBS-PS+, that process MS images separately, effectively considering the physical properties of the corresponding EO sensors in the acquisition when generating PS images. This helps prevent unrelated information from being mixed among MS images and enables accurate feature extractions. Therefore, the generated PS images have less noise and reduced color distortions than previous PAN-sharpening methods that typically fuse MS images from front-end layers. In addition, we design a novel training method and loss function to handle the problem of misregistered MS and PAN images. Our FBS-PS+ outperforms all other PAN-sharpening methods in most reference-based quality metrics, while our FBS-PS, lightweight and faster, achieves comparable quality performance to local-global transformer enhanced unfolding network (LGTEUN) that has $7.95\times $ more parameters and requires $4.06\times $ more floating-point operations per second (FLOPs).

External IDs:doi:10.1109/tgrs.2024.3523865