Teaching-playback of robot manipulator based on human gesture recognition and motion tracking

Xianmin Zhang, Huiyu Zhou, H. Cheng, Yanjiang Huang

Abstract: In this paper, we propose a new approach to teach a robot to complete a task based on the human gesture recognition and hand motion tracking. The image coordinate of the human hand and the region of interest (ROI) with the center of the hand are captured by the Kinect sensor. To recognize the human gesture and track the hand motion exactly, we calibrate the Kinect depth camera. An accuracy analysis of camera calibration was conducted by a comparison between a laser tracker and the calibrated Kinect sensor. Convex hull approach is used in the gesture recognition by comparing to other two image processing approaches. An experiment with an ABB RBI 120 robot is conducted to verify the effectiveness of the proposed methods. The experimental results demonstrate that our proposed methods can exactly recognize two kinds of gesture and track the hand motion.