Learn from Noise: Detecting Deepfakes via Regional Noise Consistency

Weiming Bai, Yufan Liu, Zhipeng Zhang, Xinyi Zhang, Bo Wang, Chengwei Peng, Weiming Hu, Bing Li

Abstract: Face forgery detection becomes increasingly crucial due to the serious security issues caused by face manipulation techniques. Various methods primarily concentrate on the features specific to certain generation techniques, potentially resulting in overfitting to the distinctive fingerprint characteristics of those manipulation techniques, thus undermining their generalizability. In contrast, our investigation reveals a prevalent phenomenon wherein regional noise consistency is disrupted during the integration of synthesized faces into source images, regardless of specific manipulation techniques. Motivated by this observation, we introduce the Regional Noise Consistency Learning Framework (RNCL), a novel approach designed to discern manipulated faces. Central to RNCL are two pivotal modules: Noise Consistency Enhancement (NCE) and Pyramidal Noise Consistency Learning (PNCL). The NCE module facilitates channel-wise and spatial-wise feature enhancement by exploiting noise inconsistencies between facial and non-facial regions. Complementarily, the PNCL module constructs a noise consistency pyramid to analyze enhanced features across multiple scales, enabling adaptive multi-scale feature integration. Leveraging the NCE and PNCL modules, our framework effectively transforms noise information into useful forgery cues, significantly enhancing forgery detection performance. Experimental results demonstrate that our method achieves state-of-the-art performance on standard benchmarks. The code will be publicly available.