Scalable and Cost-Effective Edge-Cloud Deployment for Multi-Object Tracking System

Ratul Kishore Saha, Dheeraj Chahal, Rekha Singhal, Manoj Nambiar

Abstract: The proliferation of edge devices has significantly advanced technologies in sectors such as autonomous driving and surveillance. However, deploying machine learning models on these resource-constrained devices presents challenges, including scalability and managing unpredictable workloads, which often hinder real-time performance (both latency and throughput) in edge-only environments. To address these issues, a potential approach is to deploy on an edge-cloud ecosystem, however, it may hinder latency due to communication delay between edge and cloud. We leverage the cloud service Message Queuing Telemetry Transport (MQTT) protocol powered by 5G internet, for communication to manage the latency. We propose a scalable edge-cloud ecosystem specifically designed for a Multi-Camera Sensor-based Multi-Object Tracking (MCS-MOT) pipeline within industrial deployment contexts, ensuring compliance with Service Level Agreements (SLAs). Our approach introduces a novel camera feed content-guided load balancing technique that dynamically manages workloads between edge and cloud. We extract features from incoming camera feed content to inform the load-balancing process efficiently. The load balancer determines the maximum number of concurrent feeds that can be processed at the edge, with the remaining feeds handled by the cloud based on the content of the camera feed. Additionally, we propose a cost model to estimate the expenses of deploying the edge-cloud ecosystem in real-world scenarios with dynamic workloads. Our experimental results show that the proposed SLA-compliant MCS-MOT system significantly outperforms edge-only architectures in terms of latency and throughput for the YOLO-DeepSORT algorithm. We also illustrate the estimated and actual deployment costs of the system, highlighting its cost-effective scalability and optimized real-time performance.