MeMoSORT: Memory-Assisted Filtering and Motion-Adaptive Association Metric for Multi-Person Tracking

ICLR 2026 Conference Submission18996 Authors

Keywords: Multi-Object Tracking; Memory-Assisted Kalman Filter; Motion-Adaptive Association.

TL;DR: We proposed MeMoSORT, a TBD MOT tracker with two key innovations: Memory-assisted Kalman filter, Motion-adaptive IoU.

Abstract: Multi-object tracking (MOT) in human-dominant scenarios, which involves continuously tracking multiple people within video sequences, remains a significant challenge in computer vision due to targets' complex motion and severe occlusions. Conventional tracking-by-detection methods are fundamentally limited by their reliance on Kalman filter (KF) and rigid Intersection over Union (IoU)-based association. The motion model in KF often mismatches real-world object dynamics, causing filtering errors, while rigid association struggles under occlusions, leading to identity switches or target loss. To address these issues, we propose MeMoSORT, a simple, online, and real-time MOT algorithm with two key innovations. At first, the Memory-assisted Kalman filter (MeKF) uses memory-augmented neural networks to compensate for mismatches between assumed and actual object motion. Secondly, the Motion-adaptive IoU (Mo-IoU) adaptively expands the matching region and incorporates height similarity to reduce mis-associations, while remaining lightweight. Experiments show that MeMoSORT achieves state-of-the-art performance, with HOTA scores of 67.9\% and 82.1\% on DanceTrack and SportsMOT, respectively.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 18996