Autonomous Machine Learning Workloads on Edge Devices: A Case Study

Jonathan Ezroni, Yaron Alpert

Keywords: Machine Learning, edge inference, CMSIS-NN, Deep Neural Networks, ARM Cortex-M, low-power computing.

TL;DR: A-ML capabilities allow IoT edge devices to efficiently (by applying Machine Learning) and autonomously (without cloud compute assistance) process local information and respond accordingly in various scenarios.

Abstract: By 2022 more than 20% of Internet of Things (IoT) endpoint devices are expected to have autonomous Machine Learning (AML) capabilities [1]. The A-ML capabilities allow IoT edge devices to efficiently (by applying Machine Learning) and autonomously (without cloud compute assistance) process local information and respond accordingly in various scenarios. Moreover, employing A-ML reduces power consumption and response time related to the edge device’s communication with the cloud. In this paper, we demonstrate the feasibility of integrating A-ML inference tasks on memory and performance limited (low cost) Micro Control Units (MCUs). The paper presents design tradeoffs based on quantitative performance analysis on existing low cost products.