Attention-Bayesian Hybrid Approach to Modular Multiple Particle Tracking

Abstract: Tracking multiple particles in dense scenes remains challenging due to a combinatorial explosion of trajectory hypotheses, which scales super-exponentially with the number of frames. The transformer architecture has shown a significant improvement in robustness against this high combinatorial load. However, its performance still falls short of the conventional Bayesian filtering approaches in locally sparse scenarios presenting a reduced set of trajectory hypothesis. This suggests that while transformers excel at narrowing down possible associations, they are not able to reach the optimality of the Bayesian approach in locally sparse scenario. Hence, we introduce a hybrid tracking framework that combines the ability of self-attention to learn the underlying representation of particle behavior with the reliability and interpretability of Bayesian filtering. We perform trajectory-to-detection association by solving a label prediction problem, using a transformer encoder to infer soft associations between detections across frames. This prunes the hypothesis set, enabling efficient multiple-particle tracking in Bayesian filtering framework. Our approach demonstrates improved tracking accuracy and robustness against spurious detections. These results open the way to a solution for high-clutter multiple-particle tracking scenarios that takes advantage of the large context accessible to transformers, together with the interpretability and theoretical guarantees of Bayesian filtering techniques.

Submission Type: Long submission (more than 12 pages of main content)

Changes Since Last Submission: I had unintentionally not included the corresponding author for this paper. As suggested by the editors-in-chief, I withdrew the previous submission to resubmit.

Assigned Action Editor: ~Xavier_Alameda-Pineda1

Submission Number: 8487