PACCDU: Pyramid Attention Cross-Convolutional Dual UNet for Infrared and Visible Image Fusion

Xuejiao Wang, Zhen Hua, Jinjiang Li

Abstract: Infrared and visible image fusion is an important branch in the field of information fusion, which aims to integrate effective information from different sensors and enhance the integrity of image information. Therefore, how to fully extract and retain the structural information and texture details in the source images is a pressing problem at present. In this article, a dual-branch UNet fusion network based on pyramidal attention and cross-convolution (PACCDU) is proposed to obtain fused images with high contrast, rich information, and clear contours. The network encoder uses cross-encoding blocks and pyramidal attention blocks to extract contextual features and cross-scale correlation features in different directions. The fusion block uses parallel spatial attention and channel attention to fuse feature information at different scales. The decoding stage uses large kernel convolution blocks and pyramidal attention to reconstruct the fused features. Ablation experiments and comparison experiments are conducted on the public datasets TNO, RoadScene, and NIR, and the results show that the algorithm in this article is superior in both subjective visual and objective evaluation.