R2GAN: Enhancing unseen image fusion with reconstruction-guided generative adversarial network

Abderrazak Chahi, Mohamed Kas, Ibrahim Kajo, Yassine Ruichek

Abstract: Generative Adversarial Networks (GANs) have gained prominence in computer vision, with applications that extend to image fusion. Existing fusion methods often require extensive labeled data and task-specific training, limiting their generalizability. To address these limitations, this paper presents the Reconstruction-Guided Generative Adversarial Network (R2GAN), a generic GAN-based approach designed for generic image fusion, including visible-infrared, medical, and multi-focus image fusion. The proposed R2GAN architecture consists of a primary generator to improve fusion capabilities and auxiliary generators to ensure accurate reconstruction of source image features. To optimize the model, we propose a reconstruction-guided loss function to preserve the feature distribution of the source images and improve the consistency between the fused and source images. Additionally, we introduce a semantic segmentation-guided approach to generate a comprehensive and realistic Paired Multi-Focus image dataset (PMF) to train the R2GAN model. Experimental results in multiple fusion tasks demonstrate that R2GAN delivers superior performance, outperforming state-of-the-art image fusion methods. The R2GAN framework source code is available for access on GitHub at https://github.com/CHAHI24680/R2GAN.