Contactless Camera-Based Approach for Driver Respiratory Rate Estimation in Vehicle Cabin

Walaa Othman, Alexey M. Kashevnik, Igor Ryabchikov, Nikolay Shilov

Abstract: Measuring vital signs is usually done by sensors attached to the human body. In clinical cases, the patients are being monitored by contacted devices that alert the medical staff when the patient situation becomes unstable. However, in non-clinical cases, there are situations when vital signs measurements can be used to prevent dangerous situations, like the driver monitoring task. Monitoring the driver’s vital signs has become popular for the last few years due to its significant role in preventing accidents. However, this task is challenging since contact sensors are inconvenient for the driver and can’t be used in this case. In the paper, we propose a contactless camera-based approach to calculate the respiratory rate of drivers. We suggest using the Openpose human pose estimation model to estimate the position of the chest keypoint, followed by an optical flow-based neural network (SelFlow) to calculate the keypoint displacement. After that, we clean this signal using filtering and detrending as well as count the number of peaks/troughs in a time window of one minute. We evaluated our approach in real driving conditions and it works precisely when the vehicle is stopped or moves with a speed below 3 km/h. When the vehicle moves there are a lot of additional driver motions that significantly reduce the accuracy of the respiratory rate detection. We also compared our results with the ROI approach proposed by researchers from Microsoft and concluded that the proposed approach is more accurate in vehicle cabins.