ACE: A Cross-platform and visual-Exoskeletons System for Low-Cost Dexterous Teleoperation

Shiqi Yang, Minghuan Liu, Yuzhe Qin, Runyu Ding, Jialong Li, Xuxin Cheng, Ruihan Yang, Sha Yi, Xiaolong Wang

Keywords: Teleopration System; Hardware; Imitation Learning; Robot Learning; Exoskeletons

TL;DR: A novel and powerful teleoperation system for dexterous tasks.

Abstract:

Bimanual robotic manipulation with dexterous hands has a large potential workability and a wide workspace as it follows the most natural human workflow.
Learning from human demonstrations has proven highly effective for learning a dexterous manipulation policy. To collect such data, teleoperation serves as a straightforward and efficient way to do so.
However, a cost-effective and easy-to-use teleoperation system is lacking for anthropomorphic robot hands.
To fill the deficiency, we developed \our, a cross-platform visual-exoskeleton system for low-cost dexterous teleoperation.
Our system employs a hand-facing camera to capture 3D hand poses and an exoskeleton mounted on a base that can be easily carried on users' backs. ACE captures both the hand root end-effector and hand pose in real-time and enables cross-platform operations.
We evaluate the key system parameters compared with previous teleoperation systems and show clear advantages of \our.
We then showcase the desktop and mobile versions of our system on six different robot platforms (including humanoid-hands, arm-hands, arm-gripper, and quadruped-gripper systems), and demonstrate the effectiveness of learning three difficult real-world tasks through the collected demonstration on two of them.

Video: https://www.youtube.com/watch?v=NzcePqu-LuA

Website: https://ace-teleop.github.io/

Code: https://github.com/ACETeleop/ACETeleop

Student Paper: yes

Submission Number: 13