CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion

Tobias Fischer, Yung-Hsu Yang, Suryansh Kumar, Min Sun, Fisher Yu

Keywords: Autonomous driving, 3D multi-object tracking, cross-camera fusion

TL;DR: We present CC-3DT, an effective solution to multi-camera 3D tracking with the key idea to merge 3D detections of multiple cameras into a single 3D tracking component handling both temporal and cross-camera association.

Abstract: To track the 3D locations and trajectories of the other traffic participants at any given time, modern autonomous vehicles are equipped with multiple cameras that cover the vehicle's full surroundings. Yet, camera-based 3D object tracking methods prioritize optimizing the single-camera setup and resort to post-hoc fusion in a multi-camera setup. In this paper, we propose a method for panoramic 3D object tracking, called CC-3DT, that associates and models object trajectories both temporally and across views, and improves the overall tracking consistency. In particular, our method fuses 3D detections from multiple cameras before association, reducing identity switches significantly and improving motion modeling. Our experiments on large-scale driving datasets show that fusion before association leads to a large margin of improvement over post-hoc fusion. We set a new state-of-the-art with 12.6% improvement in average multi-object tracking accuracy (AMOTA) among all camera-based methods on the competitive NuScenes 3D tracking benchmark, outperforming previously published methods by 6.5% in AMOTA with the same 3D detector.

Student First Author: yes

Website: https://www.vis.xyz/pub/cc-3dt/