Lighter is Better: Boost Your ViT in Person Re-Identification via Spatial-Aware Token Merging

ICLR 2026 Conference Submission25121 Authors

Keywords: Person re-identification, Vision transformer, Token merging, Lightweight

TL;DR: This paper proposes a training-free spatial-aware token merging paradigm for lightweight ViT in ReID, which significantly reduces computational costs while maintaining performance comparable to SOTA methods.

Abstract: Vision Transformers (ViTs) have significantly advanced person re-identification (ReID) by providing strong global modeling, but their high computational cost hinders deployment in real-time applications. Existing lightweight ReID methods mostly use token pruning, which can discard discriminative contextual information. Token merging is a moderate alternative, yet existing merging methods target image classification and overlook the local cues that ReID requires. This paper proposes STM-ReID, a spatial-aware and training-free token merging framework tailored for ViT-based lightweight ReID. STM-ReID injects information-enhanced spatial awareness into token assessment and uses the resulting scores to guide token matching and fusion, preserving identity-relevant local details while reducing computation. The framework comprises three key components: (i) DSE-Assess, a dynamic spatial-aware entropy weighting for token importance; (ii) CCF-Match, a correlation-guided matching scheme for precise pair selection; (iii) PNR-Fuse, a position response-driven computation strategy for feature aggregation. Extensive experiments on standard ReID benchmarks and general classification datasets show that STM-ReID cuts GFLOPs of the base ViT model by about 24\% while keeping accuracy comparable to state-of-the-art methods, yielding a superior accuracy–efficiency trade-off.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 25121