Local Feature Enhancement Network for Set-based Face Recognition

Ziyi Bai, Ruiping Wang, Shiguang Shan, Xilin Chen

Abstract: Set-based Face Recognition is widely applied in scenarios like law enforcement and online media data management. Compared with face recognition using a single image, the faces in the set often contain abundant appearance changes. Therefore, how to make full use of the rich information from the set and integrate them into a unified set representation become the key to set-based face recognition. Inspired by the fact that humans usually complete this fine-grained task through integrating the information from the congruent local regions (e.g. an eye to an eye) of multiple faces in a set, we propose a novel method called Local Feature Enhancement Network (LFENet), which can automatically enhance the local feature through transferring the local information across the images. Specifically, we retain the spatial semantic information of the feature maps and apply different relational functions to establish the correlation among the local features. The contained local information will be transferred to the relevant local features to enhance their discriminability. By doing so, the valuable local information carried in some local features can complement those with incomplete information. Besides, the various local information is aligned across faces under different conditions to help the model learn intra-set-compact face representations. Our method achieves state-of-the-art performances on two mainstream set-based face recognition benchmarks: IJB-A and IJB-C, which fully reflects the rationality and effectiveness of our local feature enhancement mechanism.