Novel Unified Body-Based Biometric for Clothing Invariant Human Recognition

Padmaja Jonnalagedda, Bir Bhanu

Abstract: Body-based human recognition faces increased complexity due to a myriad of challenging factors such as variations in imaging platform (ground- vs. aerial-based), diverse sensors, imaging distance, changes in clothing, arbitrary human poses, occlusion and articulation of the human body, and air turbulence. These conditions often make it difficult to discern facial features, particularly at long distances, where changes in attire also significantly contribute to recognition errors. The most commonly used RGB and shape-based features extracted from color video, suffer significant drawbacks under such conditions. To overcome these challenges, this paper introduces a novel biometric that integrates shape, appearance, and soft biometric features into a single, unified representation called the Outfit Regularizing Biometric (ORB). Through extensive experimentation, we demonstrate that the ORB representation effectively addresses some of the most significant obstacles in human recognition. We report our findings using the rigorous evaluation protocols of the BRIAR datasets (Phase 1 and 2), alongside widely used clothes-changing datasets such as PRCC, LTCC, Celeb-ReID, and VC-Clothes. Our analysis using the BRIAR dataset enables us to isolate and examine the impact of various artifacts, including changes in clothing, environmental conditions, and varying imaging distances. By employing the ORB representation, in combination with both appearance and shape-based features, and leveraging a transformer-based backbone (TransFuse), we achieve superior performance. Across different challenging scenarios, ORB consistently outperforms traditional RGB-based systems, delivering nearly an 18% improvement in accuracy on the standard BRIAR setup.

External IDs:doi:10.1109/tbiom.2025.3631607