3D-model-based face replacement in video

Yi-Ting Cheng, Virginia Tzeng, Yu Liang, Chuan-Chang Wang, Bing-Yu Chen, Yung-Yu Chuang, Ming Ouhyoung

Abstract: The development in digital technologies and the widespread Web 2.0 concept have made many digital videos accessible. Editing and modifying digital videos have become an interesting and important topic. In this paper, we present a system for face replacement in video. Most digital processing software can perform face replacement only when the poses for the source and target faces are similar, and the manipulation process with those software is often time-consuming and labor-intensive. While previous work [Blanz et al. 2004] focuses on image face replacement, our system performs face replacement in video by constructing 3D models for both target and source faces and swapping them accordingly. 3D face models are created by fitting 3D morphable models [Blanz et al. 1999] and the input is reduced to two pictures for the face to be placed in.