Generating Robot Policy Code for High-Precision and Contact-Rich Manipulation Tasks

Kaylee Burns, Ajinkya Jain, Keegan Go, Fei Xia, Michael Stark, Stefan Schaal, Karol Hausman

Primary Area: applications to robotics, autonomy, planning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: contact rich manipulation, language models as planners

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Large Language Models (LLMs) have been successful at generating robot policy code, but so far these results have been limited to high-level tasks that do not require accurate movement.
%
It is an open question how well such approaches can work for high-precision, contact-rich tasks that require controlling contact forces with the environment.
%
We find that, with the right action space, LLMs are capable of successfully generating policies for a variety of contact-rich and high-precision manipulation tasks in a zero-shot fashion.
%
Specifically, we reparameterize the action space to include robot compliance with constraints on the interaction forces and stiffnesses involved in reaching a target pose.
%
We validate this approach on subtasks derived from the Functional Manipulation Benchmark (FMB) and the IROS 2020 Robotic Grasping
and Manipulation Competition, where zero-shot policy generation in this action space improves success rates by greater than 3x and 4x, respectively, over a baseline that uses free space motions.
%
To further investigate properties that make language models well posed to generate contact-rich tasks, we also analyse language models ability to complete control-relevant arithmetic reasoning tasks over continuous numbers in-context and ablate the importance of different prompt components in generating relevant motion patterns.
Project webpage: https://dex-code-gen.github.io/dex-code-gen/

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6997