Efficient Tactile Simulation with Differentiability for Robotic Manipulation

Jie Xu, Sangwoon Kim, Tao Chen, Alberto Rodriguez Garcia, Pulkit Agrawal, Wojciech Matusik, Shinjiro Sueda

Keywords: Tactile Simulation, Tactile Manipulation, Differentiable Simulation, Sim-to-Real

TL;DR: We present an efficient simulator for simulating dense tactile force fieds wth both normal and shear components. We conduct extensitve policy learning experiments in simulation and a sim-to-real experiment on tactile insertion task.

Abstract: Efficient simulation of tactile sensors can unlock new opportunities for learning tactile-based manipulation policies in simulation and then transferring the learned policy to real systems, but fast and reliable simulators for dense tactile normal and shear force fields are still under-explored. We present a novel approach for efficiently simulating both the normal and shear tactile force field covering the entire contact surface with an arbitrary tactile sensor spatial layout. Our simulator also provides analytical gradients of the tactile forces to accelerate policy learning. We conduct extensive simulation experiments to showcase our approach and demonstrate successful zero-shot sim-to-real transfer for a high-precision peg-insertion task with high-resolution vision-based GelSlim tactile sensors.

Student First Author: yes

Website: http://tactilesim.csail.mit.edu/