Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification

yeyunchen, Jiangming Shi

Abstract: Unsupervisedvisible-infraredpersonre-identification(USL-VI-ReID) is a promising yet highly challenging retrieval task. The key challenges in USL- VI-ReID are to accurately generate pseudo-labels and establish pseudo-label cor- respondences across modalities without relying on any prior annotations. Re- cently, clustered pseudo-label methods have gained more attention in USL-VI- ReID. However, most existing methods don’t fully exploit the intra-class nu- ances, as they simply utilize a single memory that represents an identity to es- tablish cross-modality correspondences, resulting in noisy cross-modality cor- respondences. To address the problem, we propose a Multi-Memory Matching (MMM) framework for USL-VI-ReID. We first design a simple yet effective Cross-Modality Clustering (CMC) module to generate the pseudo-labels through clustering together both two modality samples. To associate cross-modality clus- tered pseudo-labels, we design a Multi-Memory Learning and Matching (MMLM) module, ensuring that optimization explicitly focuses on the nuances of individ- ual perspectives and establishes reliable cross-modality correspondences. Finally, we design a Soft Cluster-level Alignment (SCA) loss to narrow the modality gap while mitigating the effect of noisy pseudo-labels through a soft many-to- many alignment strategy. Extensive experiments on the public SYSU-MM01 and RegDB datasets demonstrate the reliability of the established cross-modality cor- respondences and the effectiveness of MMM.