Enhancing Detection of Daily-Used Face Swap Applications by Using Focused Landmark Analysis

Minh-Khoi Nguyen-Nhat, Trung-Truc Huynh-Ngo, Minh-Triet Tran, Trong-Le Do

Abstract: The rapid evolution of user-friendly, high-quality face swap apps has raised security and privacy concerns, particularly on social media. This motivates our proposal of an efficient method to identify faceswap images. Our approach is based on Vision Transformer and exploits Facial Landmark Focusing Image (FAFI), designed to target regions adjacent to facial landmarks to identify anomalies. We conduct experiments on our dataset containing diverse face-swap videos from common applications, capturing real-life contexts. Experimental results demonstrate that FAFI exhibits superior generalization potential compared to Multi-scale Retinex, especially when using the Vision Transformer as a feature extractor, with the accuracy, precision, and recall of 95.29%, 96.45%, and 93.53%, respectively. Furthermore, the visualization using Grad-CAM also demonstrates that our method with Vision Transformer tends to emphasize intricate and detailed abnormal artifacts within the face, providing promising hints to users to identify potential suspected regions in a human face for further evaluation (Code and dataset are available at https://github.com/minhkhoi1026/face-spoofing-dection).