Go to DBLP homepageTranSpike: Pixel-wise frequency reconstruction and spike interaction for remote photoplethysmography
Abstract: Facial vision-based contactless physiological signal measurement is an important intelligent application. Currently, there are a large number of fully supervised deep learning frameworks targeting this field to remotely extract biosignals by detecting facial skin optical imaging changes caused by cardiac activity. However, these solutions suffer from training data paucity, label inconsistent, and misalignment. Although many self-supervised methods have been proposed, they generally fall into the trap of capturing irrelevant periodic features and spatial semantic drifts since the very sensitive and weak pulse-induced skin color variations. To address the above issues, we design a novel self-supervised video Transformer framework for remote physiological measurements, named TranSpike. First, we abandon the popular unlabeled and blind video frame resampling, and construct positive and negative samples in pixel-wise frequency reconstruction of facial reflections. This strategy simulates pseudo heart rate labels and avoids the spike attenuation caused by existing Nyquist–Shannon and temporal random samplings. Second, we improve the vision Transformer backbone and optimize the related model to consider long-term temporal physiological cues beyond spatial context and spatiotemporal perception. Third, we propose an innovative contrastive paradigm and corresponding architecture that inspire the interaction of biological signals, pulse rates, and power spectra to recover rhythmic features. Experimental results demonstrate that our scheme outperforms representative methods when targeting mainstream datasets. Importantly, our model not only competes with existing self-supervised techniques, but also shows remarkable consistency with state-of-the-art full-supervised monitoring architectures in zero-annotation.
External IDs:dblp:journals/pr/ShaoLQHCY26
Loading