Shadow Gestures: 3D Hand Pose Estimation Using a Single Camera

Jakub Segen, Senthil Kumar

Abstract: This paper describes a system that uses a camera and a point light source to track a user's hand in three dimensions. Using depth cues obtained from projections of the hand and its shadow, the system computes the 3D position and orientation of two fingers (thumb and pointing finger). The system recognizes one dynamic and two static gestures. Recognition and pose estimation are user independent and robust. The system operates at the rate of 60 Hz and can be used as an intuitive input interface to applications that require multi-dimensional control. Examples include 3D fly-thru's, object manipulation and computer games.