Masked Face Recognition With Mask Transfer and Self-Attention Under the COVID-19 Pandemic

Meng Zhang, Rujie Liu, Daisuke Deguchi, Hiroshi Murase

Abstract: Face masks bring a new challenge to face recognition systems especially against the background of the COVID-19 pandemic. In this paper, a method used for mitigating the negative effects of mask defects on face recognition is proposed. Firstly, a low-cost, accurate method of masked face synthesis, i.e. mask transfer, is proposed for data augmentation. Secondly, an attention-aware masked face recognition (AMaskNet) is proposed to improve the performance of masked face recognition, which includes two modules: a feature extractor and a contribution estimator. Therein, the contribution estimator is employed to learn the contribution of the feature elements, thus achieving refined feature representation by simple matrix multiplications. Meanwhile, the end-to-end training strategy is utilized to optimize the entire model. Finally, a mask-aware similarity matching strategy (MS) is adopted to improve the performance in the inference stage. The experiments show that the proposed method consistently outperforms on three masked face recognition datasets: RMFRD, COX and Public-IvS. Meanwhile, a qualitative analysis experiments using CAM indicate that the contribution learned by AMaskNet is more conducive to masked face recognition.