RGBT2HS-Net: Reconstructing a Hyper-Spectral Volume from an Rgb-T Stack via an Attention-Powered Multiresolution Framework

Hao-Chiang Shao, Szu-Chi Wu, Yen-Liang Chuo, Jyun-Hao Lin, Yuan-Rong Liao, Tse-Yu Tseng

Abstract: We present our RGBT2HS-Net method, capable of reconstructing the 61-channel spectral reflectance in the visible and near-infrared spectral range from a 4-channel RGB-Thermal image stack. RGBT2HS-Net has two primary characteristics. First, it employs a multiresolution framework, allowing the input 4D RGB-Thermal image stack to be decomposed into multiresolution tensors via learnable filters for further processing. Second, it incorporates an attention mechanism with a scalable receptive field, enabling the learning of both inter-channel dependencies and intra-channel correlations for hy-perspectral reconstruction. Experiments demonstrate that our method outperforms previous state-of-the-art models in terms of the SAM score on the testing set and achieves an average PSNR greater than 27 dB on the validation set.

External IDs:dblp:conf/icassp/ShaoWCLLT24