GlandSAM: Injecting Morphology Knowledge into Segment Anything Model for Label-free Gland Segmentation

Qixiang ZHANG, Yi Li, Haonan Wang, Xiaomeng Li

Abstract:

—This paper presents a label-free gland segmentation, GlandSAM, which achieves comparable performance with supervised methods while no label is required
during its training or inference phase. We observe that
the Segment Anything model produces sub-optimal results
on gland dataset: It either over-segments a gland into
many fractions or under-segments the gland regions by
confusing many of them with the background, due to
the complex morphology of glands and lack of sufficient
labels. To address this challenge, our GlandSAM innovatively injects two clues about gland morphology into
SAM to guide the segmentation process: (1) Heterogeneity within glands and (2) Similarity with the background.
Initially, we leverage the clues to decompose the intricate glands by selectively extracting a proposal for each
gland sub-region of heterogeneous appearances. Then,
we inject the morphology clues into SAM in a fine-tuning
manner with a novel morphology-aware semantic grouping
module that explicitly groups the high-level semantics of
gland sub-regions. In this way, our GlandSAM could capture comprehensive knowledge about gland morphology,
and produce well-delineated and complete segmentation
results. Extensive experiments conducted on the GlaS
dataset and the CRAG dataset reveal that GlandSAM outperforms state-of-the-art label-free methods by a significant
margin. Notably, our GlandSAM even surpasses several
fully-supervised methods that require pixel-wise labels for
training, which highlights the remarkable performance and
potential of GlandSAM in the realm of gland segmentation.