Real-Time Respiration Monitoring of Neonates from Thermography Images Using Deep Learning

Simon Lyra, Ines Groß-Weege, Steffen Leonhardt, Markus J. Lüken

Abstract: In this work, we present an approach for non-contact automatic extraction of respiration in infants using infrared thermography video sequences, which were recorded in a neonatal intensive care unit. The respiratory signal was extracted in real-time on low-cost embedded GPUs by analyzing breathing-related temperature fluctuations in the nasal region. The automatic detection of the patient’s nose was performed using the Deep Learning-based YOLOv4-Tiny object detector. Additionally, the head was detected for movement tracking. A leave-one-out cross validation showed a mean intersection over union of 79% and a mean average precision of 93% for the detection algorithm. Since no clinical reference was provided, the extracted respiratory activity was validated for video sequences without motion artifacts using Farnebäck’s Optical Flow algorithm. A mean MAE of 8.5 breaths per minute and a mean \(\mathrm{F}_{1}\)-score of 80% for respiration detection were achieved. The model inference on NVIDIA Jetson modules showed a performance of 32 fps on the Xavier NX and 62 fps on the Xavier AGX. These outcomes showed promising results for the real-time extraction of respiratory activity from thermography recordings of neonates using Deep Learning-based techniques on embedded GPUs.