Video splicing detection and localization based on multi-level deep feature fusion and reinforcement learning

Xiao Jin, Zhen He, Jing Xu, Yongwei Wang, Yuting Su

Abstract: Splicing forgery refers to copying some regions of a video or an image to another video/image. Although image splicing detection has been studied for many years, video splicing detection has attracted relatively much less attention. In this paper, we proposed a novel framework for video splicing detection by modeling this forensic task as a video object segmentation problem. Based on the nature of this forgery operation, discontinuous noise distribution and object contours are adopted as traces to guide the localization results. The method consists of three modules: EXIF-consistency prediction, suspected region tracking, and semantic segmentation. To bridge the gap between sensor-level and semantic-level features, three modules in our framework are integrated for final tampered areas detection. Firstly, we use the EXIF-consistency prediction module to extract sensor-level traces from tampered areas. Then, we employ a deep reinforcement learning-based method for tracking suspected regions. Finally, a semantic segmentation module is adopted to localize the final results of the tampered regions. Compared with several state-of-the-art forensic approaches, our method demonstrates superiority in publicly available datasets. In terms of F1 score, our method achieves 0.623 in GRIP dataset.