DistilDIRE: A Small, Fast, Cheap and Lightweight Diffusion Synthesized Deepfake Detection

Yewon Lim, Changyeon Lee, Aerin Kim, Oren Etzioni

Keywords: Deepfake Detection, Diffusion Models

TL;DR: Inference over foundational models is slow and costly. This study shows distillation enables scalable deepfake detection. Our lightweight detector maintains performance using 97% fewer TFLOPS than DIRE, enhancing deployment and research.

Abstract:

A dramatic influx of diffusion-generated images has marked recent years, posing unique challenges to current detection technologies. While the task of identifying these images falls under binary classification, a seemingly straightforward category, the computational load is significant when employing the “reconstruction then compare” technique. This approach, known as DIRE (Diffusion Reconstruction Error), relies on inference over foundation diffusion models which is slow and expensive—limiting their practical applicability. This paper presents a case study shows how distillation can enable their more scalable use “in the wild” on a particular task (deepfake detection). To address the computational challenges and improve efficiency, we propose distilling the knowledge embedded in diffusion models to develop rapid deepfake detection models. Our approach, aimed at creating a small, fast, cheap, and lightweight diffusion synthesized deepfake detector, maintains robust performance while significantly reducing operational demands. Maintaining performance, our experimental results indicate an inference speed 3.2 times faster than the existing DIRE framework. This advance not only enhances the practicality of deploying these
systems in real-world settings but also paves the way for future research endeavors that seek to leverage diffusion model knowledge. The code and weights for our framework are available at anonymous-link.

Submission Number: 35