Multi-source Deep Domain Adaptation for Deepfake Detection

Md Shamim Seraj, Shayok Chakraborty

Abstract: With the unprecedented success of generative models like GANs, synthetic image manipulations such as deepfakes have emerged as a serious concern in today’s world. Existing techniques demonstrate promise in detecting deepfakes on which they are trained; however, their performance drops significantly when applied to detect forgeries created using other manipulation techniques, on which the model has not been sufficiently trained. Thus, detecting new types of deepfakes without losing prior knowledge about already learned faking techniques, is a problem of immense practical importance. In this paper, we propose a novel multi-source deep domain adaptation framework to address this challenge. Our framework can leverage a large amount of labeled data (fake/genuine) generated using one or more faking techniques (source domains) and a small amount of labeled data generated using a target faking technique of interest (target domain) to induce a deep neural network with good generalization capability on all the source domains, as well as the target domain. Further, our framework can efficiently utilize unlabeled data in the target domain, which is more readily available than labeled data. We design a novel loss function specific to the multi-source domain adaptation task and use the SGD method to optimize the loss and train the deep network. Our extensive empirical studies on benchmark datasets, using different types of deepfakes, corroborate the promise and potential of our framework for real-world applications. To the best of our knowledge, this is the first research effort to develop a multi-source deep domain adaptation technique for deepfake detection.