Pan-Sharpening Based on Multilevel Coupled Deep Network

Wanting Cai, Yang Xu, Zebin Wu, Hongyi Liu, Ling Qian, Zhihui Wei

Abstract: Pan-sharpening is a common image-fusion method. To improve the quality of fused images, a multilevel deep learning Pan-sharpening method is proposed in this paper. In the training phase, we introduce Coupled Sparse Denoising Autoencorder (CSDA) to reconstruct high-Resolution (HR) multispectral (MS) image from low-Resolution (LR) MS image and HR Panchromatic (Pan) image. CSDA has four networks including LM-HP network, HR-MS network, feature mapping network and fine-tuning network. The hidden features in LM-HP network and HR-MS network as well as the mapping function between the two features are learned through joint optimization. In LM-HP and HR-MS networks, the hidden features of image patch pairs are extracted by the sparse autoencoder. A sparse denoising autoencoder is used to build the nonlinear mapping between the extracted features. In the testing phase, the LR-MS and HR-Pan images patches are fed to the CSDA network to reconstruct the fused HR-MS image. The experimental results show that the proposed method is better than the traditional pans-sharpening methods.