Learning Diverse and Physically Feasible Dexterous Grasps with Generative Model and Bilevel Optimization

Albert Wu, Michelle Guo, Karen Liu

Keywords: dexterous grasping, grasp planning, bilevel optimization, generative model

TL;DR: Proposal and hardware validation of a pipeline combining generative model and bilevel optimization for generating diverse physically feasible dexterous grasps.

Abstract: To fully utilize the versatility of a multi-fingered dexterous robotic hand for executing diverse object grasps, one must consider the rich physical constraints introduced by hand-object interaction and object geometry. We propose an integrative approach of combining a generative model and a bilevel optimization (BO) to plan diverse grasp configurations on novel objects. First, a conditional variational autoencoder trained on merely six YCB objects predicts the finger placement directly from the object point cloud. The prediction is then used to seed a nonconvex BO that solves for a grasp configuration under collision, reachability, wrench closure, and friction constraints. Our method achieved an 86.7% success over 120 real world grasping trials on 20 household objects, including unseen and challenging geometries. Through quantitative empirical evaluations, we confirm that grasp configurations produced by our pipeline are indeed guaranteed to satisfy kinematic and dynamic constraints. A video summary of our results is available at youtu.be/9DTrImbN99I.

Student First Author: yes

Website: https://www.youtube.com/watch?v=9DTrImbN99I

Code: https://github.com/Stanford-TML/dex_grasp