TVFusionGAN: Thermal-Visible Image Fusion Based on Multi-level Adversarial Network Strategy

Guoyu Lu

Abstract: Thermal imaging is effective in low-light or night-time conditions due to its ability to capture thermal radiation differences, but lacks texture compared to visible images. Conversely, visible images retain more texture information, particularly during the daytime, but perform poorly at night. To address the limitations of both modalities, recent methods have utilized fusion techniques to generate images that combine thermal and visible properties. This paper presents an end-to-end fusion network leveraging generative adversarial networks (GANs) to fuse salient components from both modalities. Our network includes a generator and two discriminators. The generator produces fusion images with salient objects using a specially designed CIoU loss, while the discriminators ensure that the fused images are salient at both holistic and local scales. One discriminator encourages the fused images to resemble visible images overall, while the other ensures that targeted objects in the fused images are as salient as in thermal images. Our method effectively preserves thermal radiation of salient objects in infrared images while incorporating the textures of visible images.