Learning Equivariant Neural-Augmented Object Dynamics from Few Interactions

Sergio Orozco, Brandon B. May, Tushar Kusnur, George Konidaris, Laura Herlant

Keywords: dynamics, deformable, equivariant

TL;DR: We learn an equivariant neural-augmented particle dynamics model that learns from few interactions for robotic planning.

Abstract:

Learning data-efficient object dynamics models for robotic manipulation is challenging, especially for deformable bodies.
Popular approaches model objects as 3D graphs and learn particle displacements using graph neural networks; however, they often require thousands of interactions.
Even so, these models fail to adhere to real world physics by violating interpenetration constraints and not maintaining object shape over time.
We introduce PIEGraph, a neural-augmented dynamics model capable of learning physically-grounded object dynamics for rigid and deformable bodies from few interactions.
PIEGraph is a hierarchical framework built using two key layers: (1) a Physically Informed prior implemented as a spring mass system to model physically feasible particle motions over time, and (2) an action-conditioned Equivariant Graph Neural Network that exploits symmetries in particle motion and guides the physics prior.
We demonstrate the ability to learn object dynamics for robotic planning on ropes, cloth, stuffed animals, and rigid bodies using only a few minutes of human interaction data.

Submission Number: 8