Point-based Correspondence Estimation for Cloth Alignment and Manipulation

Mansi Agarwal, Thomas Weng, David Held

Keywords: Deformable Object Manipulation, Bimanual Manipulation, Cloth Manipulation, Optical Flow

TL;DR: We propose a novel approach for goal-conditioned cloth manipulation that combines learned correspondences and symmetry-handling techniques to align observed and goal cloth configurations before manipulation.

Abstract: Automating cloth folding is a challenging task with practical implications in various domains. Existing methods often struggle with unaligned configurations, limiting their applicability in real-world scenarios. In this research, we present FabricFlowAlignNet (FFAN), a novel approach that learns flow-based correspondences on point clouds between the current observed and goal cloth configurations. We use these learned 3D correspondences for both cloth alignment and manipulation: correspondences are used to align the observed cloth with the goal, and the flow-based correspondences are re-used as action proposals. Our experiments demonstrate that FFAN demonstrates superior performance compared to a state-of-the-art folding approach, particularly in scenarios where observed cloth is rotated or otherwise unaligned with the goal.