Vision Transformer MST++: Efficient Hyperspectral Skin Reconstruction

Ana Clara C. Silveira, Diedre Santos do Carmo, Lucas H. Ueda, Denis G. Fantinato, Paula D. P. Costa, Letícia Rittner

Abstract: Channel reconstruction transforms a subsampled multispectral image into hyperspectral, offering hyperspectral imaging benefits without a dedicated camera. MST++ is a state of the art channel reconstruction technique, but it faces memory limitations for high spatial resolution images. In this context, we introduce VITMST++, a novel architecture incorporating Vision Transformer embedding and compression, multi-resolution image context and a channel-weighted loss. Developed for the ICASSP 2024 Hyperspectral Skin Challenge, VITMST++ outperforms the state-of-the-art MST++ in both performance and computational efficiency in channel reconstruction.