Toward Autonomous Dexterous Manipulation using Diffusion Policies with a Humanoid Robot

Toby Buckley, Kevin Lynch, J. Edward Colgate

Keywords: dexterous, manipulation, robotics, autonomous, imitation learning, behavior cloning, diffusion model, diffusion policy

Abstract:

An autonomous machine learning agent was trained using demonstration data to perform a dexterous manipulation task using the Dexterity Nexus (DexNex) upper-limb robot testbed. Denoising Diffusion Probabilistic Models were used to clone the behavior of the teleoperator. The diffusion model was able to learn and perform the task, but its performance was worse than the teleoperation data it was trained from. The drop in performance is likely a combination of lack of demonstration data, limitations of the model, and slow trajectory execution. More demonstration data and more advanced trajectory execution methods are needed to realize the full potential of this technology.
This is the first demonstration of an autonomous agent controlling the DexNex hardware setup, with a task output space of 21 joint positions. This work is the start of the HAND Engineering Research Center’s effort to develop autonomous capabilities for controlling high-degree-of-freedom manipulators. Characterizing its limitations will inform future design decisions for developing dexterous systems.

Submission Number: 13