Cross-modal identity correlation mining for visible-thermal person re-identification

Sen Zhang, Zhaowei Shang, Mingliang Zhou, Yingxin Wang, Guoliang Sun

Abstract: Visible-thermal person recognition is a sub problem of image retrieval, which aims to find out the images belonging to the same pedestrian as the current image from the image set of another modality. In this paper, we propose a novel cross-modal identity correlation mining algorithm to mine potential correlation knowledge from the features of visible and thermal modalities. First, aiming at the huge visual differences caused by different imaging mechanisms, we build a correlation-enhanced knowledge transfer module based on cross-modal identity similarity to enhance the feature representation by exchanging identity knowledge between two modalities and then compress it into a shared subspace. Second, in view of different pedestrian posture and camera perspective, we design a symmetric modal-specific feature embedding module to improve the intra-modality feature discrimination, which maps the two modal images to a pair of independent feature subspaces by two fine-grained network branches. The whole algorithm can be trained in an end-to-end manner. Extensive experiments demonstrated that the proposed method outperforms the state-of-the-art methods on SYSU-MM01 and RegDB.