Embedded Real-Time Human Activity Recognition on an ESP32-S3 Microcontroller Using Ambient Audio Data

Thomas Pfitzinger, Hendrik Wöhrle

Abstract: The task of Human Activity Recognition (HAR) finds application in smart homes and assisted living and is mostly approached with machine learning. Sensor data for HAR is often collected and processed by separate devices. Performing the recognition task directly on the sensor device can decrease network traffic and reduce the related energy consumption and privacy concerns. In this work, an intelligent sensor device is developed that performs real-time HAR on an ESP32-S3 microcontroller using ambient audio data. A convolutional neural network to classify human activities is designed and subsequently reduced in size and computation cost in order to enable its inference on the microcontroller. This is achieved with neural architecture search, resulting in multiple simplified model variations. These are tested on the ESP32-S3 and a feasible version is identified.

External IDs:dblp:conf/idaacs/PfitzingerW23