Benchmarking person re-identification approaches and training datasets for practical real-world implementations

Jose Miguel Huaman Cruz, Felix Oliver Sumari Huayta, Luigy Alex Machaca Arcana, Esteban GONZALEZ CLUA, Joris Guerin

Keywords: person re-identification, benchmark study, practical deployment

Abstract: Person Re-Identification (Re-ID) is receiving a lot of attention recently. Large datasets containing labeled images of various individuals have been released, allowing researchers to develop and test many successful approaches. However, when such Re-ID models are deployed in a new city or environment, the task of searching people within a network of security cameras is likely to face an important domain shift, thus resulting in decreased performance. Indeed, while most public datasets were collected in a limited geographic area, images from a new city present different features (e.g., people's ethnicities and clothing style, weather, architecture, etc.). In addition, the whole frames of the video streams must be converted into cropped images of people using pedestrian detection models, which behave differently from the human annotators who created the dataset used for training. To better understand the extent of this issue, this paper introduces a complete methodology to evaluate Re-ID approaches and training datasets with respect to their suitability for deployment for live operations. This method is used to benchmark four Re-ID approaches and three datasets and provides interesting insight and guidelines that can help designing better Re-ID pipelines in the future.

One-sentence Summary: This paper introduces a complete methodology to evaluate Re-ID approaches and training datasets with respect to their suitability for deployment for live operations.