A General Remote Photoplethysmography Estimator with Spatiotemporal Convolutional Network

Siqi Liu, Pong C. Yuen

Abstract: Remote PPG (rPPG) has been attracting increasing attention due to its potential in a wide range of application scenarios such as clinical monitoring, physical training and face presentation attack detection. On top of manually designed solutions, the deep-learning approach appears in rPPG estimation and achieves top-level performance. However, most of them try to integrate the steps of both preprocessing and ROI selection into an end-to-end network, which limits the generalization in other applications that use different input skin regions. The ROI selection model learned on face videos may not adapt to skin regions from other body parts with different appearance and size. In this paper, we leave the preprocessing apart and propose a lightweight rPPG estimation network-DeeprPPG for general use. DeeprPPG is based on spatiotemporal convolutions and can be used as a well-defined module in wider application scenarios with different types of input skin. To further boost the robustness, a spatiotemporal rPPG aggregation strategy is designed to adaptively aggregate rPPG signals from multiple skin regions into the final one. Extensive experiments are conducted and the results illustrate its robustness when facing unseen skin regions and unseen scenarios.