ISCUTE: Instance Segmentation of Cables Using Text Embedding

Omkar Joglekar, Shir Kozlovsky, Dotan Di Castro

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Deformable Linear Objects, Instance Segmentation, Text-Promptable Models, CLIPSeg, Segment Anything Model (SAM), Instance Mask, Deep Vision-Language Fusion, Zero-Shot Generalization

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: ISCUTE advances Deformable Linear Objects (cables, wires, etc.,) instance segmentation by leveraging text embedding with CLIPSeg and SAM foundation models, achieving a record-breaking mIoU of 91.21%.

Abstract:

In the field of robotics and automation, conventional object recognition and instance segmentation methods face a formidable challenge when it comes to perceiving Deformable Linear Objects (DLOs) like wires, cables, and flexible tubes. This challenge arises primarily from the lack of distinct attributes such as shape, color, and texture, which calls for tailored solutions to achieve precise identification.
In this work, we propose a foundation model-based DLO instance segmentation technique that is text-promptable and user-friendly. Specifically, our approach combines the text-conditioned semantic segmentation capabilities of CLIPSeg model with the zero-shot generalization capabilities of Segment Anything Model (SAM). We show that our method exceeds state-of-the-art performance on DLO instance segmentation, achieving a mIoU of $91.21\%$. We also introduce a rich and diverse DLO-specific dataset for instance segmentation.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3449