An End-to-End Framework for Neonatal Asphyxia Detection Through Video Analysis

Kurnianingsih, Sou Nobukawa, Melyana Nurul Widyawati, Cipta Pramana, Nurseno Bayu Aji, Afandi Nur Aziz Thohari, Dwiana Hendrawati, Eri Sato-Shimokawara, Naoyuki Kubota

Abstract: Identifying asphyxia using computer vision in real-world settings poses challenges due to varying video quality, diverse lighting conditions, and subtle color changes in the newborn’s skin. This study presents an end-to-end framework for automated neonatal asphyxia detection using time series video analysis and makes three key contributions. First, the proposed framework integrates YOLOv8-based instance segmentation with advanced feature extraction across multiple color spaces and texture analysis to detect neonatal asphyxia through the multi-modal analysis of skin features in video streams. Second, we introduce a new quality-aware temporal analysis framework that includes adaptive quality assessment for evaluating frames in real time, multi-stage feature stability tracking across temporal windows, hysteresis-based decision logic for ensuring temporal consistency, and LightGBM classification with comprehensive feature engineering to assess severity. Third, we provide a curated time series video dataset of 12,973 frames from 45 neonates, of which some were healthy, and some had asphyxia of varying severity. The findings show that the YOLOv8-based instance segmentation achieved a mean average precision (mAP@0.5) of 0.925 for accurate skin region isolation, and the LightGBM classifier outperformed traditional models with an accuracy of 0.998 and an F1-score of 0.998. The system maintains real-time processing at 30 FPS for normal and mild asphyxia cases with a minor reduction to 20 FPS in more challenging scenarios and exhibits robust temporal stability across severity levels, with consistency scores above 0.90. This framework has the potential to enhance neonatal care through continuous monitoring and timely intervention.