SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Use

Kushal Kedia, Tyler Ga Wei Lum, Jeannette Bohg, Karen Liu

Keywords: RL, Sim2Real

TL;DR: Unify sim2real by procedural generation of tools with a single, general reward function

Abstract:

The ability to manipulate tools significantly expands the set of tasks a robot can perform. Yet, tool manipulation represents a challenging class of dexterity, requiring
grasping thin objects, in-hand object rotations, and forceful
interactions. Since collecting teleoperation data for these behaviors is challenging, sim-to-real reinforcement learning (RL)
is a promising alternative. However, prior approaches typically
require substantial engineering effort to model objects and
tune reward functions for each task. In this work, we proposeSimToolReal, taking a step towards generalizing sim-to-real RL
policies for tool manipulation. Instead of focusing on a single
object and task, we procedurally generate a large variety of
tool-like object primitives in simulation and train a single RL
policy with the universal goal of manipulating each object
to random goal poses. This approach enables SimToolRealto perform general dexterous tool manipulation at test-time
without any object or task-specific training. We demonstrate
that SimToolReal outperforms prior retargeting and fixedgrasp methods by 37% while matching the performance of
specialist RL policies trained on specific target objects and
tasks. Finally, we show that SimToolReal generalizes across
a diverse set of everyday tools, achieving strong zero-shot
performance over 120 real-world rollouts spanning 24 tasks,
12 object instances, and 6 tool categories.

Submission Number: 3