A Comprehensive Deepfake Detector Assessment Platform

Liu liu, Zhixuan Chu, Zhongjie Ba, Chengyi Yan, Ziyue Zhan, Feng Lin, Zhan Qin, Kui Ren

Keywords: deepfake detection; benchmark; evaluation

TL;DR: We propose a comprehensive evaluation framework for deepfake detection algorithms

Abstract: The rapid development of deepfake techniques has raised serious concerns about the authenticity and integrity of digital media. To combat the potential misuse of deepfakes, it is crucial to develop reliable and robust deepfake detection algorithms. In this paper, we propose a comprehensive **D**eepfake **D**etector **A**ssessment **P**latform (**DAP**), covering six critical dimensions: benchmark performance, forgery algorithm generalization, image distortion robustness, adversarial attack resilience, forgery localization accuracy, and attribute bias. Our framework aims to provide a standardized and rigorous approach to assess the performance, generalization ability, robustness, security, localization precision, and fairness of deepfake detection algorithms. Extensive experiments are conducted on multiple public and self-built databases, considering various forgery techniques, image distortions, adversarial attacks, and attributes. The proposed framework offers insights into the strengths and limitations of state-of-the-art deepfake detection algorithms and serves as a valuable tool for researchers and practitioners to develop and evaluate novel approaches in this field. All codes, scripts, and data described in this paper are open source and available at https://github.com/tempuser4567/DAP.

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3436