Learning Force Distribution Estimation for the GelSight Mini Optical Tactile Sensor based on Finite Element Analysis

Erik Helmut, Luca Dziarski, Niklas Funk, Boris Belousov, Jan Peters

Keywords: Machine Learning, Tactile Sensing, Finite Element Analysis, GelSight Mini, Force Estimation

Abstract: Contact-rich manipulation remains a major challenge in robotics. Optical tactile sensors like GelSight Mini offer a low-cost solution for contact sensing by capturing soft-body deformations of the silicone gel. However, accurately inferring shear and normal force distributions from these deformations has yet to be fully addressed. In this work, we propose a machine learning approach using a U-net architecture to predict force distributions directly from raw images. Our model, trained on force distributions inferred from FEA, demonstrates promising accuracy in predicting these distributions. It also shows potential for generalizing to a different GelSight sensor than the one used for data collection and enables real-time application. This approach holds the promise of improving tactile perception in contact-rich robotic manipulation tasks.

Submission Number: 2