Object-Level Pseudo-3D Lifting for Distance-Aware Tracking

Haoyuan Jin, Xuesong Nie, Yunfeng Yan, Xi Chen, Zhihang Zhu, Donglian Qi

Abstract: Multi-object tracking (MOT) is a pivotal task for media interpretation, where reliable motion and appearance cues are essential for cross-frame identity preservation. However, limited by the inherent perspective properties of 2D space, the crowd density and frequent occlusions in real-world scenes expose the fragility of these cues. We observe the natural advantage of objects being well-separated in high-dimensional space and propose a novel 2D MOT framework, ``Detecting-Lifting-Tracking'' (DLT). Initially, a pre-trained detector is employed to capture 2D object information. Secondly, we introduce a Mamba Distance Estimator to obtain the distances of objects to a monocular camera with temporal consistency, achieving object-level pseudo-3D lifting. Finally, we thoroughly explore distance-aware tracking via pseudo-3D information. Specifically, we introduce a Score-Distance Hierarchical Matching and Short-Long Terms Association to enhance accurate and robust association capability. Even without appearance cues, our DLT achieves state-of-the-art performance on MOT17, MOT20, and DanceTrack, demonstrating its potential to address occlusion challenges.

Primary Subject Area: [Content] Media Interpretation

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: Multi-object tracking (MOT) is a critical task for media interpretation, which supports a wide range of applications such as autonomous driving, intelligent surveillance, and human behavior prediction. To the best of our knowledge, our DLT is the first work to achieve object-level pseudo-3D lifting and comprehensively upgrade the tracking process based on distance information. Our work represents a rare exploration in the domain of 2D MOT with 3D information and can serve to encourage more researchers to delve into this direction in the future.

Submission Number: 19