Fully Tactile Dexterous Hand Grasping Strategy Combining Visual and Tactile Senses

Jian Ma, Qianqian Tian, Kuo Liu, Jixiao Liu, Shijie Guo

Abstract: Robotic dexterous hands need to rely on both visual and tactile information for grasping during grasping and interaction. The position and type of objects are identified by visual information, and stable grasping is accomplished by tactile information. In this paper, a solution based on visual-tactile fusion control is proposed. At the same time, the data set is established by the method of tactile teaching to complete stable grasping. The tactile teaching approach is inspired by adults teaching children to write by hand. The “hand-held” grip method enables a more natural and convenient grip teaching. By fusing visual and tactile information, the perceptual capability and control accuracy of the robot’s dexterous hand are improved. Using the proposed method, a series of grasping tasks were tested in an experimental environment. The results show that the method can not only effectively improve the grasping success rate of the robot dexterous hand on objects of different shapes, but also achieve stable interaction between the robot dexterous hand and the objects.