InstaBound: Instance-Boundary Signed Distance Regression

Xinzi He, Ruining Deng, Martin R. Prince, Mert R. Sabuncu

Keywords: Nuclei Segmentation, Signed Distance Field, Instance Segmentation, Deep Learning, Computational Pathology, PanNuke, NuInsSeg

TL;DR: InstaBound achieves state-of-the-art nuclei segmentation by training a standard U-Net to regress instance Signed Distance Fields (iSDF).

Abstract: Accurate instance segmentation is a cornerstone of medical image analysis, essential for tasks ranging from quantifying nuclei in histopathology to measuring cysts, metastases, and other lesions in radiology. However, these tasks are fraught with challenges due to high object density, overlapping boundaries, and significant appearance heterogeneity. In this work, we present InstaBound, a novel deep learning framework that reformulates instance segmentation as an instance boundary Signed Distance Field (iSDF) regression problem. Unlike traditional methods that rely on binary masks or bounding boxes, InstaBound learns a continuous representation of object geometry, implicitly encoding boundary information to robustly separate touching instances. We introduce a dual-task objective function combining truncated-iSDF regression with a Dice loss to ensure both structural coherence and semantic accuracy. Our experiments on the PanNuke and NuInsSeg nuclei datasets demonstrate that InstaBound achieves state-of-the-art performance, significantly outperforming established baselines such as HoVer-Net, Micro-Net, and Mask R-CNN. Specifically, we report a multi-class Panoptic Quality (mPQ) improvement of 3.4\% on the PanNuke. Our code and pre-trained models are available to facilitate further research.

Primary Subject Area: Segmentation

Secondary Subject Area: Application: Histopathology

Registration Requirement: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 57