Simultaneous Facial Action Tracking and Expression Recognition Using a Particle Filter

Fadi Dornaika, Franck Davoine

Abstract: The recognition of facial gestures and expressions in image sequences is an important and challenging problem. Most of the existing methods adopt the following paradigm. First, facial actions/features are retrieved from the images, and then facial expressions are recognized based on the retrieved temporal parameters. Unlike this main strewn, this paper introduces a new approach allowing the simultaneous recovery of facial actions and expression using a particle filter adopting, multiclass dynamics that are conditioned on the expression. For each frame in the video sequence, our approach is split in two consecutive stages. In the first stage, the 3D head pose is recovered using a deterministic registration technique based on online appearance models. In the second stage, the facial actions as well as the facial expression are simultaneously recovered using the stochastic framework with, mixed states. The proposed fast scheme is either as robust as existing ones or more robust with respect to many regards. Experimental results show the feasibility and robustness of the proposed approach