Gesture objects detection and tracking for virtual text entry keyboard interface

Kuldeep Singh Yadav, Anish Monsley K., Rabul Hussain Laskar

Abstract: Detection and tracking of the gesturing object is a vital stage in the field of dynamic gesture recognition. It becomes more challenging in the practical environment due to the variation in illumination, occlusion, pose, rotation, speed, and presence of impostors. To overcome these complexities and provides ease to the user, we designed algorithms to (i) detect and track the gesture objects (red-color marker, bare-hand); (ii) recognize the gesture. To detect the red-color marker, a region of interest-based detection model is proposed by utilizing movement and color information. This model achieves ~13% improvement over the existing baseline models. For bare hand detection, a three-degree-information algorithm is proposed by incorporating region of interest (color, movement) and AlexNet. An improvement of 14% is achieved over the baseline models. To track the bare hand, a detection and tracking algorithm is proposed by utilizing AlexNet and point-tracker. This model achieves ~10% improvement over the baseline models. Evaluation of the proposed models for detection and tracking is performed on NITS hand gesture (I-IV, VII), Oxford hand, OUHands, EgoHands databases. To recognize gesture trajectories, the deep convolutional neural network is utilized. This model is able to achieve a relative improvement of ~7% over the baseline recognition models. To evaluate the performance of the recognition model, the NITS hand gesture (I-IV, VII), MNIST, and SVHN databases are used. In addition, we introduce the NITS-Net database consisting of bare-hand, non-bare-hand images.