SmartEye: An Open Source Framework for Real-Time Video Analytics with Edge-Cloud Collaboration

Xuezhi Wang, Guanyu Gao

Abstract: Video analytics with Deep Neural Networks (DNNs) empowers many vision-based applications. However, deploying DNN models for video analytics services must address the challenges of computational capacity, service delay, and cost. Leveraging the edge-cloud collaboration to address these problems has become a growing trend. This paper provides the multimedia research community with an open source framework named SmartEye for real-time video analytics by leveraging the edge-cloud collaboration. The system consists of 1) an edge layer which enables video preprocessing, model selection, on-edge inference, and task offloading; 2) a request forwarding layer which serves as a gateway of the cloud and forwards the offloaded tasks to backend workers; and 3) a backend worker layer that processes the offloaded tasks with specified DNN models. One can easily customize the policies for preprocessing, offloading, model selection, and request forwarding. The framework can facilitate research and development in this field. The project is released as an open source project on GitHub at https://github.com/MSNLAB/SmartEye.