Real-Time Predictability Analysis and Enhancement of Deep-Learning-Based Object Tracking

Mingyang Zhang, Kristof Van Beeck, Toon Goedemé

Abstract: While Siamese object tracking has witnessed significant advancements, its hard real-time behaviour on embedded devices remains inadequately addressed. In many application cases, an embedded implementation should not only have a minimal execution latency, but this latency should ideally also be having zero variance, i.e. predictable. To bridge this gap, we firstly analyse the real-time predictability of components of a state-of-the-art deep-learning-based object video object tracking system. Our detailed experiments indicate the superiority of FPGA implementations in terms of hard real-time behaviour, but unveil important time-predictability bottlenecks. Then, we craft a dedicated hardware accelerator specifically for the bottleneck. Our method seamlessly integrates advanced tracker features and improves greatly the tracker’s speed and time-predictability on embedded systems. Implemented on a KV260 board, our quantized tracker demonstrates superior performance. These findings spotlight the immense promise of hardware acceleration in real-time object tracking and set a benchmark for forthcoming hardware-software co-design pursuits focused on achieving time-predictable object tracking.

External IDs:dblp:conf/ipmv/ZhangBG24