Occlusion robust multi-camera face tracking

Joshua Harguess, Changbo Hu, Jake K. Aggarwal

Abstract: This paper presents a novel approach to object tracking by using multiple views to assist with handling occlusion which improves the overall tracking result. The approach is applied to face tracking using a 3D cylinder head model, but any 3D rigid object may be tracked using this approach. All cameras in the system are used to estimate a joint motion model of the face, which is updated at each frame. Self-occlusion is handled by a weighted mask that depends on the pose of the face. Full face occlusion is first detected automatically by measuring and comparing image histograms of the current tracking result and a face template. If an occlusion from a camera is reported, it is not used in the global tracking result of the face from the multi-camera system. Experiments demonstrate that our method succeeds in tracking in both cases of self-occlusion and full face occlusion. Comparisons are made between single camera tracking, multi-camera tracking and occlusion robust multi-camera tracking using results from pose estimation. The performance of the occlusion robust multi-camera face tracking method is shown to produce more accurate estimates of the face pose and is able to estimate the face pose even under severe face occlusion.