A Framework for Improving Information Content of Human Demonstrations for Enabling Robots to Acquire Complex Tool Manipulation Skills

Rishabh Shukla, Omey M. Manyar, Devsmit Ranparia, Satyandra K. Gupta

Abstract: Tool manipulation is a crucial skill for robots to perform intricate tasks, and learning from demonstration methods can provide an effective means for robots to learn these skills. However, the process of collecting human demonstration data can be challenging and may lead to information loss, requiring a large number of demonstrations to learn the human's policy. In this work, we propose a novel framework for collecting information-rich human demonstration data for learning complex tool manipulation skills. Our framework can accommodate data collection from multiple modalities such as speech, gesture, motion, video, and 3D depth data. Additionally, the framework actively queries the human expert to improve the information content of the data. We showcase the effectiveness of our method in collecting demonstration data for a complex granular media transport task and performing the task on a real robot.