Hyperspectral Object Tracking With Context-Aware Learning and Category Consistency

Ye Wang, Shaohui Mei, Mingyang Ma, Yuheng Liu, Tao Gao, Huiyang Han

Abstract: Hyperspectral imaging technology is of crucial importance to improve the performance of object tracking in many remote sensing surveillance areas. Previous methods primarily focused on feature fusion strategies by employing additional enhancement modules. However, these methods commonly lack contextual understanding to distinguish the target from the background and totally ignore the category information of the targets. To address these limitations, a novel hyperspectral object tracker is proposed to incorporate context-aware learning and category consistency tracker (CCTrack), which can adaptively learn context-aware representations in hyperspectral scenarios to obtain global target information with memory storage, while constructing an interframe category consistency constraint to enhance tracking process. Specifically, CCTrack integrates an adaptive context-aware learning (ACL) mechanism, which includes a feature decoupling module (FDM) to extract specific representations from decoupled features, and a Mamba layer to retain and update long-range dependencies. To align with prior knowledge of target recognition and motion patterns, an alignment transformation module (ATM) is employed with the ACL mechanism, fully leveraging spatial-spectral representations. In addition, category consistency constraint modules (C3Ms) are introduced to enforce category consistency across frames by computing the similarities between the target features and the corresponding category name, serving as the constraint to improve tracking performance. Extensive experiments over the hyperspectral object tracking (HOT) benchmark covering various remote sensing scenarios demonstrate that CCTrack outperforms state-of-the-art methods by a significant margin.