A multi-object tracking method based on long-term and short-term hypergraph neural network matching

Wen Guo, Qigui Liu, Tuo Wang, Xinmiao Ding

Abstract: Addressing the issues of competition between detection features and Re-ID features in joint detection and embedding multi-object tracking methods, as well as difficulties in maintaining visual consistency for occluded targets in complex scenes, we propose an end-to-end hypergraph neural network matching tracking method, named HGTracker. Firstly, HGTracker introduces an enhanced Spatial Pyramid Pooling Networks (ESPPNet) module to enhance the detection capability of the target detection backbone network. This module aggregates features from different dimensions to adapt to different tasks in the tracking process, effectively alleviating the issue of competition between detection and Re-ID tasks in one-stage multi-object tracking methods. Secondly, it introduces a Short-term and Long-term Hypergraph Neural Network Matching module, which designs long-term and short-term hypergraph neural networks to associate unoccluded and occluded detection visual features. It transforms the data association problem into a hypergraph matching problem between trajectory hypergraphs and detection hypergraphs. The tracker models the relationship between trajectory segment information and the current detection frame information as a hypergraph neural network, maintaining visual trajectory consistency under severe occlusion.Through a series of comparative experiments, the proposed HGTracker tracking method, compared to the FairMOT tracking method on the MOT17 dataset, increased the HOTA value from 59.3% to 61.4%, the IDF1 value from 73.7% to 79.3%, and the MOTA value from 72.3% to 76.9%; on the MOT20 dataset, the HOTA value increased from 54.6% to 57.9%, the IDF1 value increased from 61.8% to 73.1%, and the MOTA value increased from 67.3% to 75.1%.