Hysat++: Hybrid Spectral-Wise Attention Transformer for Skin Spectral Reconstruction

Yuchen Wang, Hongyuan Wang, Jiang Xu, Chang Chen, Xue Hu, Fenglong Song, Lizhi Wang

Abstract: Hyperspectral imaging captures detailed spatial-spectral information across a wide range of wavelengths, offering a comprehensive and non-invasive method for skin analysis. In this paper, we propose a novel Hybrid Spectral-wise Attention Transformer (HySAT++) for facial skin scenarios, aimed at reconstructing skin hyperspectral images in the visible and near-infrared spectrum from facial skin multispectral data. HySAT++ simultaneously model spectral-wise similarity and particularity, and incorporates the Spectral Angle Mapper (SAM) loss as a part of the loss function. This work ranked first in the ICASSP 2024 SP Grand Challenge on Hyperspec-tral Skin Vision.