Task-Conditioned 3D U-Nets via Hypernetworks for Data-Scarce Medical Segmentation

Luca Hagen, Johanna Paula Müller, Moritz Gmeiner, Bernhard Kainz

Keywords: Segmentation, Hypernetworks, Domain Transfer, Cross Domain Generalization

TL;DR: We employ a hypernetwork to generate autnomous, fine-tunable 3D U-Nets from a single annotated example, achieving promising performance on unseen anatomical targets and approaching fully supervised models with only a few additional samples.

Abstract:

Training 3D segmentation models typically requires extensive expert annotation, which is costly and often unavailable for rare or low-prevalence pathologies. We propose a hypernetwork-based framework that amortises the prediction of parameters for compact 3D U-Nets, enabling task-specific specialisation from as little as a single annotated volume. By learning shared anatomical structure, such as coarse shape, scale, and spatial organisation, across organs and imaging modalities, the hypernetwork generates task-conditioned network parameters, allowing controlled adaptation to previously unseen but anatomically related targets without full retraining.
We evaluate the proposed approach on the CT TotalSegmentator and Medical Segmentation Decathlon benchmarks. The method achieves strong one-shot performance for anatomically homogeneous structures (e.g., liver, spleen, atrium) and demonstrates stable few-shot adaptation for more heterogeneous or low-contrast targets (e.g., tumours, prostate). In regimes with two to four annotated volumes, hypernetwork-generated U-Nets consistently outperform pretrained baselines and substantially reduce the performance gap to fully supervised models while using minimal annotation. These results indicate that weight prediction serves as an effective task-informed prior for data-scarce 3D medical image segmentation.

Primary Subject Area: Segmentation

Secondary Subject Area: Transfer Learning and Domain Adaptation

Registration Requirement: Yes

Reproducibility: ready for rebuttal, https://github.com/luca-hagen/HyperUNet

Visa & Travel: No

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 332