Face Recognition In Harsh Conditions: An Acoustic Based Approach

Yanbo Zhang, Panrong Tong, Songfan Li, Yaxiong Xie, Mo Li

Abstract: The accuracy of vision-based face recognition suffers in challenging scenarios, such as foggy or smoky weather, poor lighting, and blockage by objects like facial masks. This paper proposes an acoustic-based facial recognition system based on acoustic facial spectrum - a novel acoustic representation of human faces in 3D space. Specifically, we divide the 3D space into cubes and profile the distribution of the acoustic signal reflected by the human face inside each cube. Generating such a per-cube acoustic profile is challenging in relating each reflected signal path back to the physical location of its reflecting cube. To address the challenge, we propose a novel multipath resolving algorithm that is capable of distinguishing signal reflection happened within different cube. Based on the facial spectrum, we propose a discriminator-recognizer network that can robustly recognize human faces under varying face-microphone distances or even in presence of facial mask blockage. Extensive experimental results demonstrate that the proposed system achieves over 95% average recognition accuracy for cases with and without mask blockage. The research artifacts accompanying this paper are available via DOI: 10.5281/zenodo.11094213.