Detecting Hand Hygienic Behaviors In-the-Wild Using a Microphone and Motion Sensor on a Smartwatch

Haoyu Zhuang, Liqiang Xu, Yuuki Nishiyama, Kaoru Sezaki

Abstract: In recent years, the emergence of the COVID-19 pandemic has led to new viral variants, such as Omicron. These variants are more harmful and impose more restrictions on people’s daily hygiene habits. Therefore, during the COVID-19 pandemic, it is logical to automatically detect epidemic protective behaviors without user intent. In this study, we used multiple sensor data from an off-the-shelf smartwatch to detect several defined behaviors. To increase the utility and generalizability of the research results, we collected audio and inertial measurement unit (IMU) data from eight participants in real environments over a long period. In the model-building process, we first created a binary classification between hand hygiene behaviors(hand washing, disinfection, and face-touching) and daily behavior. Then, we distinguished between specific hand hygiene behaviors based on audio and IMU. Ultimately, our model achieves 93% classification accuracy for three behaviors(Hand washing, face touching, and disinfection). The results prove that the accuracy of the classification of behaviors has improved remarkably, which also emphasizes the feasibility of recognizing hand hygiene behaviors using inertial acoustic data.