Learning Composable Diffusion Guidance for Motion Priors

Omkar Patil, Eric Rosen, Nakul Gopalan

Track: full paper

Keywords: diffusion, composition

TL;DR: A novel method to learn diffusion models for robot action policies.

Abstract:

Diffusion models have emerged as a promising choice for learning robot skills from demonstrations. However, they face three problems: diffusion models are not sample-efficient, data is expensive to collect in robotics, and the space of tasks is combinatorially large. The established method to train diffusion models on skill demonstrations borrow from the literature on image generation, and results in a conditional distribution of robot actions given the visual, proprioceptive and other observations. However, they have little room to accommodate solutions for the aforementioned challenges, in addition to scaling the model size and paired observation-action data. In this work, we propose a novel method for training diffusion models termed ‘Composable Diffusion Guidance’ CoDiG to compositionally learn diffusion policies for robot skills. CoDiG decouples the observation modalities allowing the residual learning of one modality with respect to the others. While presenting a more intuitive modeling paradigm, CoDiG also enables the scaling of modalities such as robot motions
independently. Our preliminary results show that visual CoDiG with motion-priors outperforms the conventional way of learning visuomotor policies using diffusion models on skills with relatively low-diversity of robot motion. Further experimentation is needed to evaluate the performance and robustness of CoDiG for different observation modalities, and on different classes of skills, such as long-horizon and precise manipulation.

Presenter: ~Omkar_Patil1

Format: No, the presenting author is unable to, or unlikely to be able to, attend in person.

Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding availability would significantly influence their ability to attend the workshop in person.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 56