Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains

Marianne Rakic, Siyu Gai, Etienne Chollet, John Guttag, Adrian V Dalca

Keywords: Segmentation, Medical imaging, Foundation model

TL;DR: A new segmentation framework and sampling mechanism to produce multiple consistent label maps across similar images on held-out datasets, outperforming baselines by a wide margin.

Abstract: A single biomedical image can be segmented in multiple valid ways, depending on the application. For instance, a brain MRI may be divided according to tissue types, vascular territories, broad anatomical regions, fine-grained anatomy, or pathology. Existing automatic segmentation models typically either (1) support only a single protocol---the one they were trained on---or (2) require labor-intensive prompting to specify the desired segmentation. We introduce _Pancakes_, a framework that, given a new image from a previously unseen domain, automatically generates multi-label segmentation maps for _multiple_ plausible protocols, while maintaining semantic consistency across related images. In extensive experiments across seven previously unseen domains, _Pancakes_ consistently outperforms strong baselines, often by a wide margin, demonstrating its ability to produce diverse yet coherent segmentation maps on unseen domains.

Primary Area: Machine learning for sciences (e.g. climate, health, life sciences, physics, social sciences)

Submission Number: 9218