B2MFuse: A Bi-Branch Multiscale Infrared and Visible Image Fusion Network Based on Joint Semantics Injection

Published: 01 Jan 2024, Last Modified: 05 Mar 2025IEEE Trans. Instrum. Meas. 2024EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: Infrared and visible image fusion (IVIF) is a key technique for synthesizing a comprehensive representation of a scene by exploiting diverse perceptual information. However, existing fusion methods encounter challenges in simultaneously preserving intricate texture details and extracting high-level semantic information—essential for downstream vision tasks. To address these issues, this article presents bi-branch multiscale infrared and visible image fusion network (B2MFuse), a novel bi-branch multiscale IVIF network based on joint semantics injection. The bi-branch consists of an interactive detail branch and a parallel semantic branch, both featuring dual paths for infrared and visible modality. The former employs a channel exchange strategy that maximizes the capture of modality-specific details while obtaining complementary features from the alternate modality. The latter efficiently captures semantic information and provides flexible scene knowledge guidance to the interactive detail feature extraction branch (ID-branch), facilitating the subsequent top-to-bottom multiscale feature fusion and reconstruction process. A spatial weighted channel attention fusion module (SWCAFM) is then meticulously designed to enhance the integration of crucial fine-grained features across different scales. Furthermore, a scene-perception loss function is tailored to account for variations in the original image content. The synergy between B2MFuse’s advanced architecture and loss function ensures robust and superior fusion results in diverse environments, in particular enhancing human visual observation and supporting downstream visual tasks. Extensive evaluations on four public datasets demonstrate the superiority of our B2MFuse, compared with the state-of-the-art (SOTA) IVIF methods. The source code is available at: https://github.com/arkymeng/B2MFuse.
Loading