Towards Generalizable Multi-Object Tracking

Zheng Qin, Le Wang, Sanping Zhou, Panpan Fu, Gang Hua, Wei Tang

Abstract: Multi-Object Tracking (MOT) encompasses various tracking scenarios, each characterized by unique traits. Ef-fective trackers should demonstrate a high degree of gen-eralizability across diverse scenarios. However, existing trackers struggle to accommodate all aspects or necessi-tate hypothesis and experimentation to customize the asso-ciation information (motion and/or appearance) for a given scenario, leading to narrowly tailored solutions with limited generalizability. In this paper, we investigate the factors that influence trackers' generalization to different scenar-ios and concretize them into a set of tracking scenario at-tributes to guide the design of more generalizable trackers. Furthermore, we propose a “point-wise to instance-wise relation” framework for MOT, i.e., GeneralTrack, which can generalize across diverse scenarios while eliminating the need to balance motion and appearance. Thanks to its supe-rior generalizability, our proposed GeneralTrack achieves state-of-the-art performance on multiple benchmarks and demonstrates the potential for domain generalization.