Facial feature tracking combining model-based and model-free method

Jianyu Wang, Wen Gao, Shiguang Shan, XiaoPeng Hu

Abstract: We propose a novel facial feature tracker, which integrates model-free and model-based methods to track dense facial features reliably with complex non-rigid motions in low quality video sequences. The tracker consists of the mouth motion model part and the enhanced KLT (Kanade-Lucas-Tomasi) tracker part. There are three key elements in our algorithms: dense facial feature tracking; noise removal with global rank constraints; characteristic non-rigid motion description. Experiments show good results on tracking dense facial features under various expressions, even when some facial features have degenerate features.