FaceClone: Interactive Facial Shape and Motion Cloning System Using Multi-View Images

Kyungjune Lee, Jeonghaeng Lee, Hyucksang Lee, Mingyu Jang, Seongmin Lee, Sanghoon Lee

Abstract: We present an interactive face cloning system that imitates users' facial expressions in virtual space. To do this, we construct a multi-view capture studio. Our system consists of offline and online stages. In the offline stage, facial shapes and light conditions are jointly optimized to obtain a realistic facial scan from static multi-view images. In the online stage, the facial scan is parameterized using landmarks and it is deformed according to the users' facial expressions. Furthermore, our system provides a GUI to allow user interaction with the generated facial clone. We demonstrate our system enables immersive communication in a virtual world.