All You Need Is A Reference: Cross-modality Referring Segmentation for Abdominal MRI
Keywords: promptable segmentation model, cross-modality, referring segmentation
Abstract: Multi-modality MRI scans can provide comprehensive diagnoses of abdominal disease but this also introduces new segmentation burdens to derive quantitative imaging biomarkers. In this work, we propose a referring segmentation task where users only need to draw simple scribbles on one modality, called reference modality, to guide the segmentation of both the unseen target modalities and the reference modality. To benchmark the multi-modality segmentation task, we provide a new dataset with 3,277 organs from 534 MRI scans, covering five commonly used MRI modalities. Furthermore, we present a referring segmentation model, CrossMR, to simultaneously segment multiple modalities based on scribbles on reference modality. Experiments demonstrate that our method can achieve comparable performance to the state of the art on one in-distribution reference modality and significantly better generalization ability on four out-of-distribution modalities. This opens a door for efficiently segmenting targets across multiple modalities. The new dataset, code, and trained model weights will be publicly available at https://ref-seg-mr.github.io/.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 12337
Loading