Integral Probability Metrics for Perceptual Learning in Generative Cross-Modal Person Re-Identification

Nirmala Murali, Deepak Mishra

Abstract: Person Re-Identification (Person-ReID) is the problem of recognizing an identity in various instances across various cameras. Cross-modal Person-ReID extends this task to match images of different modalities, posing a significant challenge due to the considerable gap between modalities. The availability of the exact RGB-IR pairs for each identity and pose will help the system understand the feature space better. Thus, a generative model leveraging Optimal Transport Theory is proposed to synthesize IR images corresponding to available RGB images, enhancing the training data for the Person-ReID model. These images can be considered as distributions, and finding out how each distribution differs will eventually tell the model how each identity varies. Comparing the feature vectors using the conventional distance metrics might only work for some cases. Hence, this is done using the Integral Probability metrics, which finds the difference between two probability distributions by bringing in perceptual similarity while also aligning the inter-modality images. Additionally, a part feature attention module is proposed to learn the essential features in every RGB-IR pair. This method combines various loss functions based on Integral Probability metrics, including Wasserstein distance and Maximum Mean Discrepancy. The proposed method showed significant improvements in the cross-modal Person-ReID result.

External IDs:doi:10.1007/978-3-031-78341-8_30