Deep Correspondence Learning for Effective Robotic Teleoperation using Virtual Reality

Sanket Gaurav, Zainab Al-Qurashi, Amey Barapatre, George Maratos, Tejas Sarma, Brian D. Ziebart

Abstract: By projecting into a 3-D workspace, robotic teleoperation using virtual reality allows for a more intuitive method of control for the operator than using a 2-D view from the robot's visual sensors. This paper investigates a setup that places the teleoperator in a virtual representation of the robot's environment and develops a deep learning based architecture modeling the correspondence between the operator's movements in the virtual space and joint angles for a humanoid robot using data collected from a series of demonstrations. We evaluate the correspondence model's performance in a pick-and - place teleoperation experiment.