HLF-Net: Pansharpening Based on High- and Low-Frequency Fusion Networks

Wenxiu Diao, Feng Zhang, Haitao Wang, Wenbo Wan, Jian De Sun, Kai Zhang

Abstract: Many deep neural networks (DNNs) have been constructed for the pansharpening task. However, the differences between the high and low frequencies in images are not considered in some DNN-based pansharpening methods. As high and low frequencies have different information of images, it is difficult for the same network to learn and reconcile the two kinds of frequencies. Considering the aforementioned differences, we propose a new pansharpening network to fuse the high and low frequencies in low spatial resolution multispectral (LRMS) and panchromatic (PAN) images separately. Specifically, a high- and low-frequency fusion network (HLF-Net) is constructed, which is composed of a high-frequency fusion network (HF-Net) and a low-frequency fusion network (LF-Net). In the HF-Net, skip attention is introduced into U-Net to better retain the high frequencies in feature maps. The LF-Net uses the involution to capture the dependency among the channels of feature maps. Experiments on the GeoEye-1 dataset reveal that the proposed network outperforms some state-of-the-art methods. The code can be accessed at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/RSMagneto/HLF-Net</uri>