Probabilistic Contact Mode Planning for Multi-Finger Manipulation using Diffusion Models

Thomas Power, Abhinav Kumar, Fan Yang, Sergio Francisco Aguilera Marinovic, Soshi Iba, Rana Soltani Zarrin, Dmitry Berenson

Keywords: Diffusion Models, Dexterous Manipulation, Manipulation Planning

TL;DR: We use a diffusion model to generate trajectories used in graph-search based planning of contact mode sequences for multi-finger manipulation

Abstract:

Planning contact-rich interactions for multi-finger manipulation is challenging due to the high-dimensionality and hybrid nature of dynamics.
Recent advances in data-driven methods have shown promise, but are sensitive to the quality of training data.
Combining learning with classical methods like trajectory optimization and search adds additional structure to the problem and domain knowledge in the form of constraints, which can lead to outperforming the data on which models are trained.
We present Diffusion-Informed Probabilistic Contact Search (DIPS), which uses an A* search to plan a sequence of contact modes informed by a diffusion model.
Our method uses a particle filter-inspired method to reason about variability in diffusion sampling arising from model error, estimating likelihoods of trajectories using a learned discriminator.
We show that our method outperforms ablations that do not reason about variability and can plan contact sequences that outperform those found in training data across multiple tasks.
We evaluate on simulated tabletop card sliding and screwdriver turning tasks,
as well as the screwdriver task in hardware to show that our combined learning and planning approach transfers to the real world.

Submission Number: 32