Learning dual aggregate features for face forgery detection

Yuru Kou, Qian Jiang, Jun Zhang, Xin Jin, Ping Wei, Shengfa Miao, Xing Chu

Abstract: The rapid development of face forgery technologies brings security issues, and the importance of face forgery detection has increased. While some existing methods deliver satisfactory results when both training and testing occur on the same dataset, these methods struggle to generalize across unseen forgery datasets. Some works consider extracting unseen forgeries in terms of high-frequency information for judgment, but the lack of synergistic global considerations for such forgeries and features of the original RGB image tends to result in overfitting to specific local textures, making it difficult to further improve the generalization. In this work, to address this challenge, we propose a novel two-stream CNN-based face forgery detector. This detector synergistically combines RGB features with global high-frequency constrained forgery features, enhancing the effectiveness of face forgery detection. To this end, we design three components: 1) Multi-scale Aggregated Constrained Convolution (MACC) module. It creates modalities that both preserve comprehensive information and accentuate forgery traces. 2) Dual Spatial Aggregation Enhance (DSAE) module, which globally and synergistically aggregates and enhances features from both streams. 3) Dual Channel Enhance Aggregation (DCEA) module, which harmonizes the information across the two-stream channels based on high correlation between the streams and performs mutual enhancement. Our experimental results demonstrate that our method excels in face forgery detection, thereby achieving an AUC of \(99.63\%\) on the FF++ dataset, surpassing the existing state-of-the-art two-stream network-based detection methods.