Illumination Variation-Resistant Network for Heart Rate Measurement by Exploring RGB and MSR Spaces

Lili Liu, Zhaoqiang Xia, Xiaobiao Zhang, Xiaoyi Feng, Guoying Zhao

Abstract: Remote photoplethysmography (rPPG) is an essential way of monitoring the physiological indicator heart rate (HR), which has important guiding significance for preventing and controlling cardiovascular diseases. However, most existing HR measurement approaches require ideal illumination conditions, and the illumination variation in a realistic situation is complicated. In view of this issue, this article proposes a robust HR measurement method to reduce performance degradation due to unstable illumination in facial videos. Specifically, two complementary color spaces [RGB and multiscale retinex (MSR)] are abundantly utilized by exploring the potential of space-shared information and space-specific characteristics. Subsequently, the time-space Transformer with sequential feature aggregation (TST-SFA) is exploited to extract physiological signal features. In addition, a novel optimization strategy for model learning, including affinity variation, discrepancy, and task losses, is proposed to train the whole algorithm in an end-to-end manner jointly. Experimental results on three public datasets show that our proposed method outperforms other approaches and can achieve more accurate HR measurement under different illuminations. The code will be released at https://github.com/Llili314/IRHrNet.