Iterative Interactive Modeling for Knotting Plastic Bags

Chongkai Gao, Zekun Li, Haichuan Gao, Feng Chen

Keywords: Plastic Bag Manipulation, Learning from Demonstrations

TL;DR: Knotting plastic bags randomly dropped from the air with a dual-arm robotic system and iterative interactive modeling.

Abstract: Deformable object manipulation has great research significance for the robotic community and numerous applications in daily life. In this work, we study how to knot plastic bags that are randomly dropped from the air with a dual-arm robot based on image input. The complex initial configuration and terrible material properties of plastic bags pose challenges to reliable perception and planning. Directly knotting it from random initial states is difficult. To tackle this problem, we propose Iterative Interactive Modeling (IIM) to first adjust the plastic bag to a standing pose with imitation learning to establish a high-confidence keypoint skeleton model, then perform a set of learned motion primitives to knot it. We leverage spatial action maps to accomplish the iterative pick-and-place action and a graph convolutional network to evaluate the adjusted pose during the IIM process. In experiments, we achieve an 85.0% success rate in knotting 4 different plastic bags, including one with no demonstration.

Student First Author: yes