GuideFuse: A Novel Guided Auto-Encoder Fusion Network for Infrared and Visible Images

Zeyang Zhang, Hui Li, Tianyang Xu, Xiaojun Wu, Yu Fu

Abstract: Although the deep network has rich semantic expression ability, the details of the source image will inevitably be lost due to the increase of model depth. Thus, how to introduce the image details into a deep network is a key problem for image fusion tasks. To solve this problem, in this article, we propose a novel gradient-based auto-encoder fusion network for infrared and visible images, termed GuideFuse. We calculate the gradient map of the source image, and a special GuideValue (GV) cooperating with it to guide the Decoder to reconstruct the image. Then, a novel fusion strategy based on the calculated GV is proposed. The whole training process can be described as training an auto-encoder, in which the Encoder is responsible for extracting the features of the source image as much as possible. At the same time, the decoder reconstructs the source image according to the extracted features. Experimental results show that the proposed fusion network achieves the best fusion effect compared with the existing fusion methods.