In-Hand Gravitational Pivoting Using Tactile SensingDownload PDF

16 Jun 2022, 10:45 (modified: 11 Nov 2022, 03:45)CoRL 2022 PosterReaders: Everyone
Student First Author: yes
Keywords: Tactile Pose Sensing, In Hand Manipulation
TL;DR: We propose a learning-based system for in-hand manipulation using gravitational pivoting
Abstract: We study gravitational pivoting, a constrained version of in-hand manipulation, where we aim to control the rotation of an object around the grip point of a parallel gripper. To achieve this, instead of controlling the gripper to avoid slip, we \emph{embrace slip} to allow the object to rotate in-hand. We collect two real-world datasets, a static tracking dataset and a controller-in-the-loop dataset, both annotated with object angle and angular velocity labels. Both datasets contain force-based tactile information on ten different household objects. We train an LSTM model to predict the angular position and velocity of the held object from purely tactile data. We integrate this model with a controller that opens and closes the gripper allowing the object to rotate to desired relative angles. We conduct real-world experiments where the robot is tasked to achieve a relative target angle. We show that our approach outperforms a sliding-window based MLP in a zero-shot generalization setting with unseen objects. Furthermore, we show a 16.6\% improvement in performance when the LSTM model is fine-tuned on a small set of data collected with both the LSTM model and the controller in-the-loop. Code and videos are available at
Supplementary Material: zip
13 Replies