GATER: Learning Grasp-Action-Target Embeddings and Relations for Task-Specific Grasping

Ming Sun, Yue Gao

Abstract: Intelligent service robots require the ability to perform a variety of tasks in dynamic environments. Despite the significant progress in robotic grasping, it is still a challenge for robots to decide grasping position when given different tasks in unstructured real life environments. In order to overcome this challenge, creating a proper knowledge representation framework is the key. Unlike the previous work, in this letter, task is defined as a triplet including grasping tool, desired action and target object. Our proposed algorithm GATER (Grasp–Action–Target Embeddings and Relations) models the relationship among grasping tools–action–target objects in embedding space. To validate our method, a novel dataset is created for task-specific grasping. GATER is trained on the new dataset and achieve task-specific grasping inference with 94.6% success rate. Finally, the effectiveness of GATER algorithm is tested on a real service robot platform. GATER algorithm has its potential in human behavior prediction and human-robot interaction.