Scale Your Dataset Without Robot

Kyungzun Rim, Jiwon Kim, Ilkwon Hong, SuHyun Yoon

Keywords: Imitation Learning, Virtual Reality, Hand Demonstrations

TL;DR: We expand small teleoperation datasets with VR hand demos, bridging gaps via post-processing and achieving stronger policies.

Abstract: Imitation learning for robotic manipulation requires extensive demonstration data, yet traditional teleoperation methods are time-consuming, physically constrained, and produce biased datasets. We present a novel VR-based data collection pipeline that addresses these limitations by capturing natural hand demonstrations without robot control. Our approach transforms VR-tracked hand poses into robot-executable trajectories through automated post-processing. We evaluated our method on a real-world task using a Franka Panda manipulator. While the teleoperation-only dataset achieved only 20\% success rate due to limited coverage and small dataset size, augmenting it with hand-collected episodes resulted in the combined dataset achieving 63\% success rate-a threefold improvement. Notably, our merged dataset matches dual-camera policy performance using only single-image input. Our results demonstrate that VR-based hand demonstrations provide an accessible, efficient solution for scaling robot learning datasets while improving policy generalization and task performance.

Submission Number: 22