Generating Deepfakes with Stable Diffusion, ControlNet, and LoRA

Download PDF
Open Webpage

Stefano Bistarelli, Francesco Santini, Edoardo Toma Tavassi

Published: 01 Jan 2025, Last Modified: 24 Sept 2025ARES (Workshops 3) 2025EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: We propose a different approach to generate deepfake videos based on Stable Diffusion, ControlNet, and Low-Rank Adaptation (LoRA). Stable Diffusion offers us greater control and fine-tuning options in the generation process. Compared to GANs, the proposed technique enables quick and easy modification of the obtained video by using a text prompt, adding or removing details, and altering the style and context of the deepfake. We describe the approach used and the generation pipeline, and then we show the application interface developed for the generation. Finally, we compare the quality of our deepfake generation framework with two other related approaches using two different tools that detect video/image manipulations.