Preliminary Analysis on the Usage of Hyperspectral Reconstruction for Imaging Photoplethysmography and Heart Rate Detection

Tobias Vogelsang, Alexander Woyczyk, Filip Packan, Jonas Kolonko, Alwin Hoffmann, Björn Schuller, Shahin Amiriparian, Sebastian Zaunseder

Abstract: Hyperspectral imaging (HSI) of the skin enables far-reaching diagnostic statements concerning anatomical and physiological aspects. However, hyperspectral cameras are expensive and have limitations with regard to spatial and temporal resolution. Hyperspectral reconstruction provides a means to transfer RGB data to a hyperspectral representation, potentially overcoming limitations of equipment for HSI. This contribution investigates whether a state-of-the-art deep learning (DL) technique is usable to transform RGB videos to a hyperspectral representation and if such representation can be used to extract the blood volume pulse (BVP) and heart rate (HR). Our results indicate that the chosen DL technique performs well on the reconstruction task using the Hyper-Skin database. At the same time, the physiological information is preserved. E.g. with respect to HR extraction in own experimental data, using the original green channel yields a correlation coefficient of 0.81 to a reference HR. When using a synthesized green channel from the DL reconstruction, the correlation even rises to 0.93. Using a regression-based approach for hyperspectral reconstruction, we achieved a correlation of 0.92. Our findings indicate the potential of using hyperspectral reconstruction to yield physiological information from videos. Future works will focus on dedicated methods to process the reconstructed hyperspectral data to exploit the full potential of the pursued approach.