Towards integrated tactile sensorimotor control in anthropomorphic soft robotic hands

Nathan F. Lepora, Chris Ford, Andrew Stinchcombe, Alfred Brown, John Lloyd, Manuel G. Catalano, Matteo Bianchi, Benjamin Ward-Cherrier

Abstract: In this work, we report on how a sense of touch can be used to control an underactuated anthropomorphic robot hand, based on an integration that respects the hand’s mechanical functionality. Our focus is on integrating the sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot hand designed around the principle of adaptive synergies, with the BRL tactile fingertip (TacTip), a soft biomimetic optical tactile sensor. We consider: (i) closed-loop tactile control to establish a light contact on an unknown held object, based on the structural similarity of the tactile image; and (ii) controlling the estimated pose of a held object, using a convolutional neural network approach developed for other TacTip sensors. Accurate control was found for a range of hard and soft objects (to sub-millimetre accuracy and a few degrees). Overall, this gives a foundation to endow soft robotic hands with human-like touch, with implications for autonomous grasping, manipulation, human-robot interaction and prosthetics.