Multimodal Breathing Rate Estimation Using Facial Motion and RPPG From RGB Camera

Migyeong Gwak, Korosh Vatanparvar, Li Zhu, Nafiul Rashid, Mohsin Y. Ahmed, Jungmok Bae, Jilong Kuang, Alex Gao

Abstract: Camera-based respiratory monitoring is contactless, non-invasive, unobtrusive, and easily accessible compared to conventional wearable devices. This paper presents a novel multimodal approach to estimating breathing rate based on tracking the movement and color changes of the face through an RGB camera. A machine learning model determines the final breathing rate between two separately calculated ones from breathing motion and remote photoplethysmography (rPPG) to improve the measurement performance in a broader range of breathing frequencies. Our proposed pipeline is evaluated with 140 facial video recordings from 22 healthy subjects, including 6 controlled and 2 spontaneous breathing tasks ranging from 5 to 30 BPM. The estimation accuracy achieves 1.33 BPM mean absolute error and 86.53% pass rate within 2 BPM error criteria. To the best of our knowledge, our approach outperforms previous works that use a face region alone with a single RGB camera.