An End-to-end Efficient Framework for Remote Physiological Signal Sensing

Chengyang Hu, Ke-Yue Zhang, Taiping Yao, Shouhong Ding, Jilin Li, Feiyue Huang, Lizhuang Ma

Abstract: Remote photoplethysmography (rPPG) is utilized to estimate the heart activities from videos, which has drawn great interest from both researchers and companies recently. Many existing rPPG deep-learning based approaches focus on measuring the average heart rate (HR) from facial videos, which do not provide enough detailed information for many applications. To recover more detailed rPPG signals for the challenge on Remote Physiological Signal Sensing (RePSS), we propose an end-to-end efficient framework, which measures the average heart rate and estimates corresponding Blood Volume Pulse (BVP) curves simultaneously. For efficiently extracting features containing rPPG information, we adopt the temporal and spatial convolution as Feature Extractor, which alleviates the cost of calculation. Then, BVP Estimation Network estimates the frame-level BVP signal based on the feature maps via a simple 1DCNN. To improve the learning of BVP Estimation Net-work, we further introduce Heartbeat Measuring Network to predict the video-level HR based on global rPPG information. These two networks facilitate each other via super-vising Feature Extractor from different level to promote the accuracy of BVP signal and HR. The proposed method obtains the score 168.08 (M <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IBI</inf> ), winning the third place in this challenge.