Remote Blood Oxygen Estimation From Videos Using Neural Networks

Joshua Mathew, Xin Tian, Chau-Wai Wong, Simon Ho, Donald K. Milton, Min Wu

Abstract: Peripheral blood oxygen saturation (SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> ) is an essential indicator of respiratory functionality and received increasing attention during the COVID-19 pandemic. Clinical findings show that COVID-19 patients can have significantly low SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> before any obvious symptoms. Measuring an individual's SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> without having to come into contact with the person can lower the risk of cross contamination and blood circulation problems. The prevalence of smartphones has motivated researchers to investigate methods for monitoring SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> using smartphone cameras. Most prior schemes involving smartphones are contact-based: They require using a fingertip to cover the phone's camera and the nearby light source to capture reemitted light from the illuminated tissue. In this paper, we propose the first convolutional neural network based noncontact SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> estimation scheme using smartphone cameras. The scheme analyzes the videos of an individual's hand for physiological sensing, which is convenient and comfortable for users and can protect their privacy and allow for keeping face masks on. We design explainable neural network architectures inspired by the optophysiological models for SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> measurement and demonstrate the explainability by visualizing the weights for channel combination. Our proposed models outperform the state-of-the-art model that is designed for contact-based SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> measurement, showing the potential of the proposed method to contribute to public health. We also analyze the impact of skin type and the side of a hand on SpO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> estimation performance.