Manipulation Concept: Towards Deriving Generalizable and Physics-informed Manipulation Knowledge of Articulated Objects

Yixuan Jiang, Jiude Wei, Cewu Lu, Jianhua Sun

Keywords: Object Manipulation, Robot Cognition

Abstract: Gripper-based articulated object manipulation requires robots to reason about both object structure and the physical constraints of grippers, yet prior work has paid little consideration to grippers’ unique characteristics and their interaction with object structures. To alleviate this gap, we introduce **Manipulation Concept**, a novel analytic representation that encodes gripper manipulation skills as parameterized program templates. Each concept formalizes the interaction between a specific actionable part featuring structure and semantic (*e.g.*, cuboid door, ring handle) and a gripper action (*e.g.*, push, lift), linking geometries and semantics with executable robot actions. Building on this representation, we develop an end-to-end framework that (i) leverages a vision-language model to select the most suitable concept for the actionable part, (ii) estimates geometric and affordance parameters to instantiate the selected concept and ground it in the physical world, and (iii) generates precise gripper-specific actions to complete the task. Extensive experiments in simulation and real world demonstrate that our method outperforms prior approaches in both accuracy and generalization, achieving stronger generalization across object categories, and reliable execution in manipulation tasks.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 1343