Analysing the Interplay of Vision and Touch for Dexterous Insertion Tasks

Janis Lenz, Theo Gruner, Daniel Palenicek, Tim Schneider, Inga Pfenning, Jan Peters

Keywords: dexterous insertion, tactile sensing, reinforcement learning

TL;DR: This study shows that integrating tactile feedback with vision significantly improves robustness in challenging robotic insertion tasks and provides insights into designing effective multi-modal control systems.

Abstract: Robotic insertion tasks remain challenging due to uncertainties in perception and the need for precise control, particularly in unstructured environments. While humans seamlessly combine vision and touch for such tasks, effectively integrating these modalities in robotic systems is still an open problem. Our work presents an extensive analysis of the interplay between visual and tactile feedback during dexterous insertion tasks, showing that tactile sensing can greatly enhance success rates on challenging insertions with tight tolerances and varied hole orientations that vision alone cannot solve. These findings provide valuable insights for designing more effective multi-modal robotic control systems and highlight the critical role of tactile feedback in contact-rich manipulation tasks.

Submission Number: 41