RGB-Guided Spectral Image Generation through GANs with Implicit Spectral Learning

Emmanuel Martinez, Roman Jacome, Andrés Jerez, Henry Arguello

Abstract: Spectral imaging is a technique that enables the acquisition and analysis of radiation emitted by incident light from a scene. Spectral acquisition involves various scanning strategies, such as snapshot spectral image acquisition based on compressive sensing theory. However, scanning methods require meticulous calibration processes in the optical setups and pose challenges in implementation, especially in uncontrolled environments. Current research has employed generative adversarial networks (GANs) to produce new spectral images and alleviate reliance on complex optical setups. In addition, traditional methods are focused on RGB-to-spectral mapping techniques, where new spectral images are not created. Therefore, this work proposes spectral imaging generation through RGB image dataset guidance by using GANs. Specifically, a generative model can produce spectral images from random noise input and map them to RGB images through a spectral response matrix, which is fed into a discriminator model for adversarial training. To ensure realistic spectral image generation, an implicit learning approach to spectral information is introduced, where a pretrained model with spectral images is used to regularize the generated spectral images during training. Finally, a post-processing step normalizes the mean and standard deviation of generated spectral images according to each spectral band of the real training spectral image dataset. The generated spectral images are validated as a data augmentation strategy by performing spectral image reconstruction based on compressive sensing and using RGB images.