Variation-Net: Interpretable Variation-Inspired Deep Network for Pansharpening

Kun Li, Wei Zhang, Xin Tian, Jiayi Ma, Huabing Zhou, Zhongyuan Wang

Abstract: In this study, we propose Variation-net, an interpretable variation-inspired deep network for pansharpening, which aims to fuse panchromatic (PAN) and multispectral (MS) images for a high-resolution MS image. We first construct a novel variational pan-sharpening model with clear physical meanings. As the relationship between the PAN and MS images in the real situation is complex and nonlinear, we explore the similarity between PAN and MS images from the sparsity of nonlinear transforms in this variational pansharpening model. As a result, spatial details can be accurately transferred from PAN image to MS image. Furthermore, we build the Variation-net by unrolling the iterative shrinkage-thresholding algorithm to solve the proposed variational pansharpening model. Therefore, all modules in Variation-net have clear physical meanings and are easily observed, leading to good generalization capability. Meanwhile, nonlinear transforms and other parameters in the variational pansharpening model are learned end–to–end. The experiments demonstrate that Variation-net outperforms the state-of-the-art methods from the aspects of visual effect and objective quality analysis.