Development and validation of a deep learning-based algorithm for drowsiness detection in facial photographs

Syed Sameed Husain, Junaid Mir, Syed Muhammad Anwar, Waqas Rafique, Muhammad Obaid Ullah

Abstract: Drowsiness is a feeling of sleepiness before the sleep onset and has severe implications from a safety perspective for the individuals involved in industrial activities, mining, and driving. The state-of-the-art computer vision (CV) based drowsiness detection methods generally utilize multiple deep convolutional neural networks (DCNN) without investigating deep feature aggregation techniques for the drowsiness detection task. More importantly, the reported results are mostly based on acted drowsy data, making the utilization of models trained on such data highly arguable for detecting drowsiness in real-life situations. Towards ameliorating this, we first present a comprehensive real drowsy data curated from 50 subjects, where subjects are labeled as fresh or drowsy. Further, four DCNN models: Xception, ResNet101, InceptionV4, and ResNext101, are trained on our dataset using transfer learning to select a baseline model for our drowsiness detection method. Moreover, an experimental study is performed using five different pooling methods: global max, global average, generalized mean, region of interest, and Weibull activation, to compute a robust and discriminative global descriptor. Our results reveal that the parametric Weibull activation pooling is the best suited for aggregating deep convolutional features. Additionally, a low complexity model based on the MobileNetV2 is proposed for a deployable drowsiness detection solution in mobile devices. The detection accuracy of 93.80% and 90.50% is achieved using our proposed Weibull-based ResNext101 and MobileNetV2 models, respectively. Moreover, our results show that the proposed non-invasive method outperforms the polysomnography signals-based invasive drowsiness detection approach.