Geometric Shape Reasoning for Zero-Shot Task-Oriented Grasping

Samuel Li, Sarthak Bhagat, Joseph Campbell, Yaqi Xie, Woojun Kim, Katia P. Sycara, Simon Stepputtis

Keywords: 3D Object Reasoning, Grasping and Manipulation

TL;DR: We present a novel zero-shot task-oriented grasping method, representing an object with a symbolic graph created from decomposed shape primitives that enables fine-grained semantic reasoning via a LLM for effective real-world task-oriented grasping.

Abstract: We present a novel zero-shot task-oriented grasping method leveraging a geometric decomposition of a target object into shape primitives that we represent in a graph structure. Our approach employs only the object name and task along with this graph to engage the commonsense reasoning capabilities of large language models to dynamically assign semantic meaning and subsequently task suitability to each decomposed part. Through extensive real-world robotic experiments, we demonstrate that our approach is capable of identifying the correct part in 92% and successfully lifting the object in 82% of the tasks we evaluate.

Submission Number: 14