Dynamic Mixed-Prototype Model for Incremental Deepfake Detection
Abstract: The rapid advancement of deepfake technology poses significant threats to social trust. Although recent deepfake detectors have exhibited promising results on deepfakes of the same type as those present in training, their effectiveness degrades significantly on novel deepfakes crafted by unseen algorithms due to the gap in forgery patterns. Some studies have enhanced detectors by adapting to the continuously emerging deepfakes through incremental learning. Despite the progress, they overlooked the scarcity of novel samples that can easily lead to insufficient learning of forgery patterns. To mitigate this issue, we introduce the Dynamic Mixed-Prototype (DMP) model, which dynamically increases prototypes to adapt to novel deepfakes efficiently. Specifically, the DMP model adopts multiple prototypes to represent both real and fake classes, enabling learning novel patterns by expanding prototypes and jointly retaining knowledge learned in previous prototypes. Furthermore, we propose the Prototype-Guided Replay strategy and Prototype Representation Distillation loss, both of which effectively prevent forgetting learned knowledge based on the prototypical representation of samples. Our method surpasses existing incremental deepfake detectors across four datasets and exhibits superior generalizability to novel deepfakes through learning limited deepfake samples.
Primary Subject Area: [Content] Vision and Language
Relevance To Conference: This work contributes to multimedia/multimodal processing by detecting deepfake multimedia with a new model, the Dynamic Mixed-Prototype (DMP). The DMP model uses multiple prototypes to learn and represent the evolving patterns of deepfakes, allowing it to incorporate new fake examples without forgetting old ones. With a specialized replay strategy and distillation loss, it maintains knowledge of past deepfakes while adapting to new ones. The model's ability to generalize from few examples and outperform other detectors on various datasets marks a significant advancement in the fight against deepfake technology.
Submission Number: 806
Loading