Light field collaborative perception for visual object tracking

Mianzhao Wang, Fan Shi, Xu Cheng, Meng Zhao

Abstract: Applying powerful appearance models to locate moving targets is gaining popularity. However, distracting elements in complex scenes often lead to deviations in target perception. Compared with appearance cues, explicit 4D spatial-angular cues provided by light field is more beneficial for addressing this problem, but they are far from being fully explored in existing visual trackers. In this paper, we propose a collaboration graph-based light field collaborative perception network (LFCPNet) that decomposes the boundary perception to mine light field spatial-angular cues. Furthermore, we propose an adaptive propagation method to deploy appearance and spatial-angular cues in tracking. By assessing the adaptive score from the propagated tracking status, the tracker can effectively combine multi-modal cues from light field to distinguish similar objects. Our tracking method is evaluated using R8TRACK, a light field tracking dataset that we collect. The experiments show that our method outperforms the state-of-the-art methods and significantly improves the tracking performance.

External IDs:doi:10.1016/j.patcog.2025.112196