Visual Appearance and Soft Biometrics Fusion for Person Re-Identification Using Deep Learning

Samee Ullah Khan, Noman Khan, Tanveer Hussain, Khan Muhammad, Mohammad Hijji, Javier Del Ser, Sung Wook Baik

Abstract: Learning descriptions of individual pedestrian is a common goal of both person re-identification (P-ReID) and attribute recognition methods, which are typically differentiated only in terms of their granularity. However, existing P-ReID methods only consider identification labels for individual pedestrian. In this article, we present a multi-scale pyramid attention ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MSPA</i> ) model for P-ReID that jointly manipulates the complementarity between semantic attributes and visual appearance to address this limitation. The proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MSPA</i> method mainly comprises three steps. Initially, a backbone model followed by appearance and attribute networks is individually trained to perform P-ReID and pedestrian attribute classification tasks. The attribute network primarily focuses on suppressed image areas associated with soft biometric data while retaining the semantic context among attributes using a convolutional long short-term memory architecture. Additionally, the identification network extracts rich contextual features from an image at varying scales using a residual pyramid module. In the second step, the dual network features are fused, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MSPA</i> is re-trained for the P-ReID task to further improve its complementary capabilities. Finally, we experimentally evaluated the proposed model on the two benchmark datasets Market-1501 and DukeMTMC-reID, and the results show that our approach achieved state-of-the-art performance.