Beyond the Screen: Evaluating Deepfake Detectors under Moiré Pattern Effects

Razaib Tariq, Minji Heo, Simon S. Woo, Shahroz Tariq

Abstract: The detection of deepfakes is crucial for mitigating the societal impact of falsified video content. Despite the development of various algorithms for this purpose, challenges arise for detectors in real-world scenarios, especially when users capture deepfake content from screens and upload it online or when detectors operate on external devices like smartphones, requiring the capture of potential deepfakes through the camera for evaluation. A significant challenge in these scenarios is the presence of Moiré patterns, which degrade image quality and complicate conventional classification methods, notably deep neural networks (DNNs). However, the impact of Moiré patterns on the effectiveness of deepfake detection systems has not been adequately explored. This study aims to investigate how capturing deepfake videos via digital screen cameras affects the accuracy of detection mechanisms. We introduced the Moiré patterns by capturing the display of a monitor using a smartphone camera and conducted empirical evaluations using four widely recognized datasets: CelebDF, DFD, DFDC, and FF++. We compare the performance of twelve SOTA detectors on deepfake videos captured under the influence of Moiré patterns. Our findings reveal a performance decrease of up to 33.1 and 31.3 percentage points for image- and video-based detectors. Therefore, highlighting the challenges posed by Moiré patterns and other naturally induced artifacts is critical for improving the effectiveness of real-world deepfake detection efforts. To facilitate further research, we will release the Moiré pattern impact version of CelebDF, DFD, DFDC, and FF++ datasets with this paper. Our code is available here: https://github.com/Razaib-Tariq/deepmoire