Dynamics based 3D skeletal hand tracking

Stan Melax, Leonid Keselman, Sterling Orsten

Abstract: Natural human computer interaction motivates hand tracking research, preferably without requiring the user to wear special hardware or markers. Ideally, a hand tracking solution would provide not only points of interest, but the full state of an entire hand. [Oikonomidis et al. 2011] demonstrated a particle swarm optimization that tracked a 3D skeletal hand model from a single depth camera, albeit using significant computing resources. In contrast, we track the hand from a single depth camera using an efficient physical simulation, which incrementally updates a model's fit and explores alternative candidate poses based on a variety of heuristics. Our approach enables real-time, robust 3D skeletal tracking of a user's hand, while utilizing a single x86 CPU core for processing.