TaDFusion: Infrared and Visible Image Fusion Network Based on The Target Detection Task-driven Method

Shaohui Jin, Qixu Liu, Hao Liu, Mingliang Xu

Abstract: The combination of visible and infrared images is intended to facilitate complex vision tasks by combining target information and rich texture. By focusing solely on visual perception enhancement, current fusion algorithms do not take into account performance on high-level vision tasks. As a solution to these problems, this research develops a high-level vision task-driven image fusion network (TaDFusion) that combines image fusion and target identification tasks. Through cascading of the image fusion and target detection modules we can significantly improve the performance of advanced vision tasks by using detection loss to guide the information back to the image fusion module. Our algorithm provides better texture preservation and pixel intensity distribution than existing methods based on extensive comparisons and generalization experiments. Besides our framework demonstrates the greatest advantages in facilitating advanced vision tasks by not only generating visually appealing fused images but also detecting higher mAPs than state-of-the-art methods, according to a comparison of the performance of various fusion algorithms in target detection tasks.