EM Enhancement of 3D Head Pose Estimated by Perspective Invariance

Jian-Gang Wang, Eric Sung, Ronda Venkateswarlu

Abstract: In this paper, a new approach is proposed for estimating 3D head pose from a monocular image. The approach assumes the more difficult full perspective projection camera model as against most previous approaches that approximate the non-linear perspective projection via linear affine assumption. Perspective-invariance is used to estimate the head pose from a face image. Our approach employs a general prior knowledge of face structure and the corresponding geometrical constraints provided by the location of a certain vanishing point to determine the pose of human faces. To achieve this, eye-lines, formed from the far and near eye corners, and mouth-line of the mouth corners are assumed parallel in 3D space. Then the vanishing point of these parallel lines found by the intersection of the eye-line and mouth-line in the image can be used to infer the 3D orientation and location of the human face. Perspective invariance of cross ratio and harmonic range are used to locate the vanishing point stably. In order to deal with the variance of the facial model parameters, e.g. ratio between the eye-line and the mouth line, an EM framework is applied to update the parameters iteratively. We use the EM strategy to first compute the 3D pose using some initially learned (PCA) parameters, e.g. ratio and length, then update iteratively the parameters for individual persons and their facial expressions till convergence. The EM technique models data uncertainty as Gaussian defined over positions and orientation of facial plane. The resulting weighted parameters estimation problem is solved using the Levenberg-Marquardt method. The robustness analysis of the algorithm with synthetic data and real face images are included.