A VM/Containerized Approach for Scaling TinyML Applications

Meelis Lootus, Kartik Thakore, Sam Leroux, Geert Trooskens, Akshay Sharma, Holly Ly

Keywords: TinyML, Virtualization, Containerization, Security, Bytecode, WebAssembly, Distributed Computing, Orchestration, TinyContainer, Zero Trust

TL;DR: We introduce a set of open-source tools that make it easy to deploy, update and monitor machine learning models on a wide variety of edge devices.

Abstract: Although deep neural networks are typically computationally expensive to use, technological advances in both the design of hardware platforms and of neural network architectures, have made it possible to use powerful models on edge devices. To enable widespread adoption of edge based machine learning, we introduce a set of open-source tools that make it easy to deploy, update and monitor machine learning models on a wide variety of edge devices. Our tools bring the concept of containerization to the TinyML world. We propose to package ML and application logic as containers called Runes to deploy onto edge devices. The containerization allows us to target a fragmented Internet-of-Things (IoT) ecosystem by providing a common platform for Runes to run across devices.