Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulation

Published: 29 Oct 2024, Last Modified: 03 Nov 2024CoRL 2024 Workshop MRM-D PosterEveryoneRevisionsBibTeXCC BY 4.0
Keywords: Robot-gated DAgger, Diffusion Policy
TL;DR: Diff-DAgger enhances Diffusion Policy through data aggregation, effectively handling multi-modality in demonstration data. Unlike Ensemble DAgger, it provides a more reliable approach for a robot's self-assessment of uncertainty.
Abstract: Abstract: Diffusion policy has shown impressive results in handling multi-modal tasks in robotic manipulation. However, it has fundamental limitations in out-of- distribution failures that persist due to compounding errors and its limited capa- bility to extrapolate. One way to address these limitations is robot-gated DAgger, an interactive imitation learning with a robot query system to actively seek ex- pert help during policy rollout. While robot-gated DAgger has high potential for learning at scale, existing methods like Ensemble-DAgger struggle with highly expressive policies: They often misinterpret policy disagreements as uncertainty at multi-modal decision points. To address this, we introduce Diff-DAgger, an ef- ficient robot-gated DAgger algorithm that leverages the training objective of dif- fusion policy. We evaluate Diff-DAgger across different robot tasks and show that Diff-DAgger overall achieves 81.5% accuracy on the task failure prediction and improves task completion rate by 13.8%. We hope that this work opens up a path for efficiently incorporating expressive policies into interactive robot learning.
Submission Number: 43
Loading