Hand action detection from ego-centric depth sequences with error-correcting Hough transform

Chi Xu, Lakshmi Narasimhan Govindarajan, Li Cheng

Abstract: Highlights • An effective and efficient solution for hand action detection from mobile egocentric depth sequences. • A novel error- correcting mechanism to tackle the bottleneck issue of incorrect votes from Hough transform. • A comprehensive, in-house annotated ego-centric hand action dataset on which the proposed method is carefully evaluated. • A deep learning baseline. Abstract Detecting hand actions from ego-centric depth sequences is a practically challenging problem, owing mostly to the complex and dexterous nature of hand articulations as well as non-stationary camera motion. We address this problem via a Hough transform based approach coupled with a discriminatively learned error-correcting component to tackle the well known issue of incorrect votes from the Hough transform. In this framework, local parts vote collectively for the start & end positions of each action over time. We also construct an in-house annotated dataset. Our system is empirically evaluated on this real-life dataset as well as a synthetic dataset, where it is shown to deliver favorable results in real-time (around 112 frame-per-second). To facilitate reproduction, the new dataset and our implementation are also provided online.