NeuroRenderedFake: A Challenging Benchmark to Detect Fake Images Generated by Advanced Neural Rendering Methods

Chengdong Dong, Vijayakumar Bhagavatula, Zhenyu Zhou, Ajay Kumar

Keywords: DeepFake Detection, Biometrics, Forensics, Neural Rendered Deepfake, Hyper-Realistic Fakes

TL;DR: A Challenging Benchmark for Detecting Fake Images Generated by Advanced Neural Rendering Methods

Abstract: The remarkable progress in neural-network-driven visual data generation, especially with neural rendering techniques like Neural Radiance Fields and 3D Gaussian splatting, offers a powerful alternative to GANs and diffusion models. These methods can generate high-fidelity images and lifelike avatars, highlighting the need for robust detection methods. However, the lack of any large dataset containing images from neural rendering methods becomes a bottleneck for the detection of such sophisticated fake images. To address this limitation, we introduce NeuroRenderedFake, a comprehensive benchmark for evaluating emerging fake image detection methods. Our key contributions are threefold: (1) A large-scale dataset of fake images synthesized using state-of-the-art neural rendering techniques, significantly expanding the scope of fake image detection beyond generative models; (2) A cross-domain evaluation protocol designed to assess the domain gap and common artifacts between generative and neural rendering-based fake images; and (3) An in-depth spectral energy analysis that reveals how frequency domain characteristics influence the performance of fake image detectors. We train representative detectors, based on spatial, spectral, and multimodal architectures, on fake images generated by both generative and neural rendering models. We evaluate these detectors on 15 groups of fake images synthesized by cutting-edge neural rendering models, generative models, and combined methods that can exhibit artifacts from both domains. Additionally, we provide insightful findings through detailed experiments on degraded fake image detection and the impact of spectral features, aiming to advance research in this critical area.

Dataset URL: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/BRUMCG

Code URL: https://dataverse.harvard.edu/previewurl.xhtml?token=ad8b3242-17bb-483a-8682-4b5f317aa58f

Primary Area: Applications of Datasets & Benchmarks for in Creative AI

Submission Number: 2387