Spectral-Aware Multi-Object Tracking in Harsh Aerial Perception Domains

Leandro Di Bella, Joseph Mimassi, Leon Denis, Adrian Munteanu

Keywords: Multi-Object Tracking, UAV, Robotic, Hyperspectral data

TL;DR: We present SAT, a training-free framework that injects multi-band signal into the association step of a multi-object tracker.

Abstract: Perception in harsh robotics domains, including low-altitude aerial, underwater, and space, relies increasingly on multi-band sensors, yet the spectral signal is routinely discarded at the tracking stage. We argue this is a missed opportunity: material-dependent reflectance differences that are invisible in RGB imagery persist even when targets subtend only 10--20 pixels, making them the most reliable identity cue available in these regimes. We present SAT, a training-free tracker that injects an 8-band spectral descriptor directly into the data-association cost, with zero learned parameters in the spectral pathway. On the Multispectral Multi-Object Tracking (MMOT) benchmark (8 bands, 395-950nm), SAT achieves Higher Order Tracking Accuracy (HOTA) 55.8 online (+2.2 over BoT-SORT) and 56.5 offline (+2.9), outperforming every training-time modification we tested. A formal analysis derives a O(1/\sqrt{N}) bound on the spectral sampling noise that predicts a sharp sensitivity cliff, and a descriptor sweep reveals that even an 8-D per-band mean captures the full spectral gain, suggesting a fundamental information limit when targets are small relative to sensor resolution.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Paper Acceptance: No

Submission Number: 27