TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

Shivin Dass, Wensi Ai, Yuqian Jiang, Samik Singh, Jiaheng Hu, Ruohan Zhang, Peter Stone, Ben Abbatematteo, Roberto Martín-Martín

Keywords: Mobile Manipulation, Imitation Learning, Robot Teleoperation

TL;DR: We present a modular teleoperation interface capable of incorporating a variety of input modalities for teleoperation of mobile manipulators and showcase it's efficacy in collecting high quality demonstrations for imitation learning.

Abstract: A critical bottleneck limiting imitation learning in robotics is the lack of data. This problem is more severe in mobile manipulation, where collecting demonstrations is harder than in stationary manipulation due to the lack of available and easy-to-use teleoperation interfaces. In this work, we demonstrate TeleMoMa, a general and modular interface for whole-body teleoperation of mobile manipulators. TeleMoMa unifies multiple human interfaces including RGB and depth cameras, virtual reality controllers, keyboard, joysticks, etc., and any combination thereof. We demonstrate the versatility of TeleMoMa by teleoperating several existing mobile manipulators — PAL Tiago++, Toyota HSR, and Fetch — in simulation and the real world. We demonstrate the quality of the demonstrations collected with TeleMoMa by training imitation learning policies for mobile manipulation tasks involving synchronized whole-body motion. With a user study we demonstrate the importance of TeleMoMa’s modularity. For more information and video results, robin-lab.cs.utexas.edu/telemoma-web/.

Submission Number: 22