RTAT: A Robust Two-Stage Association Tracker for Multi-object Tracking

Song Guo, Rujie Liu, Narishige Abe

Abstract: Data association is an essential part in the tracking-by-detection based Multi-Object Tracking (MOT). Most trackers focus on designing a better data association strategy to improve the tracking performance. The rule-based handcrafted association methods are simple and highly efficient but lack generalization capability to deal with complex scenes. While the learnt association methods can learn high-order contextual information to deal with various complex scenes, but they have the limitations of higher complexity and cost. To address these limitations, we propose a Robust Two-stage Association Tracker, named RTAT, where the first-stage association is performed between tracklets and detections to generate tracklets with high purity, and the second-stage association is performed between tracklets to form final trajectories. For the first-stage association, we use a simple data association strategy to generate tracklets with high purity by setting a low threshold for the matching cost in the assignment process. For the second-stage association, we adopt the message-passing GNN framework, which models the tracklet association as a series of edge classification problem in hierarchical graphs, so that it can recursively merge short tracklets into longer ones. Our tracker RTAT ranks first on the test set of MOT17 and MOT20 benchmarks in most of the main MOT metrics: HOTA, IDF1, and AssA. More specifically, RTAT achieve 67.2 HOTA, 84.7 IDF1, and 69.7 AssA on MOT17, and 66.2 HOTA, 82.5 IDF1, and 68.2 AssA on MOT20.