Is PEFT Enough for Cell Segmentation? An Empirical No-Go Result on Frozen Foundation Models

Joonsuk Hwang, Hyowon Woo, Hyobin Seok, Jangwon Seo, Jeong Hoon Lee

Keywords: Cell segmentation, Foundation model, Parameter-Efficient Fine-Tuning (PEFT), Cellpose-SAM, LiveCELL

TL;DR: Adapting generic foundation models via parameter-efficient fine-tuning (PEFT) reveals inherent limitations in precise cell instance segmentation, underscoring the need for domain-native models.

Abstract: Since the results of cell segmentation are used as inputs to various downstream biological analyses such as cell count, morphology, and density, the accuracy of each individual cell instance is important, and the practical utility cannot be sufficiently evaluated by average segmentation scores alone. This study analyzes the effects and limitations of parameter-efficient fine-tuning (PEFT) for SAM-family foundation cell segmentation models, using Cellpose-SAM as a case study. On LiveCELL, we compared not only adapter-based fine-tuning but also reweighting losses and Cell separation losses, and observed that the test $AP_{COCO}$ of all settings converged to a narrow range of approximately 0.555-0.562, starting from 0.5447 of Cellpose-SAM (Base), and the same was consistently observed in the high-IoU region (AP@0.9). This consistent phenomenon suggests that the ceiling in this setting may be related to the backbone representation itself rather than to the capacity or structure of the PEFT modules. Based on this observation, this study proposes that, beyond PEFT approaches that attach external modules on top of generic segmentation foundation models, foundation models designed for the cell biology domain itself need to be examined.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 88