LLRFaceFormer: Lightweight Face Transformer for Real-World Low-Resolution Recognition

Yaozhe Song, Chaoyi Wang, Hongying Tang, Songrui Han, Mingchi Li, Guanjun Tong

Abstract: Recent deep learning-based face recognition (FR) methods have demonstrated remarkable performance in high-resolution (HR) or down-sampled low-resolution (LR) tasks. However, these methods often exhibit disappointing speed-accuracy tradeoffs when deployed on real-world LR scenarios due to limited model generalization. To this end, we propose a lightweight Face Transformer framework for real-world LRFR named LLRFaceFormer. First, we propose a Transformers-as-Convolutions (TaC) network using a Transformer layer to replace the matrix multiplication in the standard convolutional process. This hybrid approach combines the strengths of Transformers and CNNs, allowing the TaC network to extract sufficient effective identity information via a global receptive field while adaptively discarding redundant homogeneous identity information on constructed LR faces. Second, we propose a Transformer-specific adaptive average procedure that incorporates tensor shape operations and a depthwise (DW) convolution. This procedure enables the LLRFaceFormer framework to focus on different regions of the input images. We also introduce an identity-aware simulator that generates real-world-like blurred LR scenarios during the training process to reduce the distribution discrepancy between the training LR faces and the testing real-world LR faces. The identity-aware simulator is simultaneously trained with the FR network with a cooperative training strategy. Further experiments illustrate the significantly superior speed-accuracy tradeoffs over existing LRFR methods with state-of-the-art (SOTA) performance on several LRFR benchmarks.