A Dataset of Human Knot Tying Demonstrations with Paired Rope Topology and Hand Trajectories

Youngbin Yoon, Ayoung Hong

Attend In Person: Yes

Keywords: knot tying, learning from demonstration, deformable objects

TL;DR: We present a stage-structured dataset of human knot tying demonstrations that pairs hand motion with rope topology to study knot tying as a sequence of topology transitions.

Abstract:

Learning from demonstration (LfD) for deformable linear objects such as ropes requires more than imitating hand trajectories. Grasping positions, substep order, and motions vary across demonstrators, yet rope topology at stage boundaries converges to consistent states. To study knot tying conditioned on topology, we present a small but structured dataset of 30 human demonstrations of the Overhand and Figure-Eight knots, tied by five participants. Each session is segmented into stages by brief hand removal, yielding rope-only frames for clean topology snapshots. For every stage, we provide (i) 3D fingertip keypoints, (ii) rope crossings with depth-based over/under labels, and (iii) synchronized RGB-D imagery.
We also include demonstrations with separated strands to provide clean cases, while real tying often produces overlaps and self-occlusions that challenge current algorithms. Rather than a large benchmark, this dataset offers a starting point for studying knot tying as a sequence of topology transitions and for developing policies that go beyond trajectory imitation.

Submission Number: 21