Enhanced multi-object tracking via embedded graph matching and differentiable Sinkhorn assignment: addressing challenges in occlusion and varying object appearances

Yajuan Zhang, Yongquan Liang, Junjie Wang, Houying Zhu, Zhihui Wang

Abstract: In the realm of computer vision, the duty of multiple objects tracking remains challenging, especially in scenarios involving occlusions and varying object appearances. In this work, we propose an innovative approach leveraging embedded graph matching to address these challenges. The proposed method constructs separate detection and tracklet graphs, to capture contextual relationships and matching constraints. An embedded graph matching network is employed to encode higher-order structural information into vertex features, significantly improving robustness against the cases of occlusions. Incorporating a differentiable Sinkhorn layer enables efficient optimal assignment, enhancing computational efficiency. Our experiments on MOT16, MOT17, and MOT20 datasets demonstrate competitive performance of the proposed method, contributing to smart city surveillance, autonomous driving, and other real-time tracking applications. Here, we achieved a 57.1% MOTA score on MOT17, highlighting the effectiveness of our proposed method.