Robust Teleoperation of an Anthropomorphic Robotic Hand through Gesture Classification and Continuous Mapping

Cheng Pan, Kai Junge, Josie Hughes

Keywords: Robust Teleoperation, Dexterous Manipulation, Anthropomorphic Hand, Data Collection

TL;DR: This paper has been published in IEEE Robotics & Automation Magazine (2026)

Abstract:

Teleoperation allows for control of an anthropomorphic robot hand by leveraging the human ability to perform complex manipulation tasks.
However, the mismatch between the human and robot motions affects the intuitiveness and performance of teleoperation tasks.
We propose a hybrid Gesture-based mapping method, where a classifier first identifies a gesture from a predefined discrete set, and then a regression model determines the continuous gesture motion progression.
The discrete nature of the classification allows for the robot to execute a predefined motion, matching the intention of the human, while the regression model allows for a continuous and thus smooth and intuitive experience.
To benchmark the proposed method, we have also developed a baseline method which directly maps each human finger motion to the robot.
We systematically evaluate the two methods by varying the operator to include those with varying experience levels and hand sizes, introducing signal delay of zero or one second, and testing across 20 object geometries.
The proposed method outperforms the baseline through a 29\% decrease in average grasping time for an expert (with and without a signal delay), an 34\% increase in success rate, and 64\% increase in System Usability Scale (SUS) scores for novice users.

Submission Number: 24