RAW2HSI: Learning-Based Hyperspectral Image Reconstruction from Low-Resolution Noisy Raw-RGB

Shushik Avagyan, Vladimir Katkovnik, Karen O. Egiazarian

Abstract: In this paper, the problem of generating (hallucinating) a high-resolution hyperspectral image from a single low-resolution raw-RGB image is considered. To solve this problem, a general learning-based framework is proposed. It consists of two modules: a data adaptation module, and a backbone, deep feature extraction module. The data adaptation module is a shallow network consisting of pixel shuffling/unshuffling and shallow feature extraction. The deep feature extraction module which is an inherent part of many spectral reconstruction networks, aims at spectral super-resolution. Different spectral reconstruction networks have been studied as the backbone modules in the proposed framework. As a result of extensive simulations, it has been demonstrated that the proposed solution significantly outperforms the sequential approach of combining several state-of-the-art methods of image demosaicing, denoising, spatial and spectral super-resolution (by up to 6 dB in PSNR), and has large savings in the computational complexity (by over 5 times) with respect to the sequential method.