P3TFusion: Progressive Two-Stage Infrared and Visible Image Fusion Network Focused on Enhancing Target and Texture Information

Shupan Li, Yuan Zhang, Xiaoheng Jiang, Mingyuan Jiu, Yang Lu, Li Chen, Wei Guo, Yan Chen, Ming Dong, Mingliang Xu

Abstract: Infrared and visible image fusion strives to create images with enhanced information by seamlessly integrating complementary data from two distinct modalities. However, current progressive image fusion methods encounter a challenge where visible texture details, infrared salient targets, and low-light background information often blend into each other, compromising the clarity of texture and target details. To tackle this challenge, we introduce a progressive two-stage infrared and visible image fusion network focused on enhancing target and texture information, named P3TFusion. In the initial fusion stage, we devise a progressive multi-feature feedback adjustment network (PMFANet) that adaptively integrates dual-branch feature information and performs feedback adjustment to optimize the extraction of texture and target features, ensuring the preservation of more beneficial information. In the subsequent fusion stage, we incorporate a novel embedded texture preference enhancement network (ETPENet), which specializes in capturing fine-grained information of texture preferences by reusing the visible image while effectively suppressing the interference generated by infrared and visible images captured under varying lighting environments. Ultimately, by fine-tuning the loss function, we guarantee that the fused image preserves optimal texture information, color distribution, contrast, and brightness. P3TFusion has been rigorously trained and tested on the LLVIP dataset and further subjected to generalization experiments on the TNO, MSRS, and RoadScene datasets. Compared to state-of-the-art methods, our approach yields superior quantitative results across key evaluation metrics (EN, AG, SF, SD) and stands out in qualitative assessments for its ability to preserve intricate texture and target details.