Advancing Medical Image Segmentation with Self-Supervised Learning: A 3D Student-Teacher Approach for Cardiac and Neurological Imaging

Moona Mazher, Daniel C. Alexander, Abdul Qayyum, Steven A Niederer

Keywords: Self-Supervised Learning (SSL), Whole Heart Segmentation (WHS), Ischemic Stroke Lesion Segmentation (ISLES), CT Imaging, MRI Imaging, Cardiac Imaging, Neurological Imaging, xLSTM, Multi-Modal Imaging, Traumatic Brain Injury (TBI).

TL;DR: Introducing a self-supervised 3D student-teacher framework with xLSTM for improved cardiac and neurological image segmentation, addressing data scarcity and modality variability to enhance clinical decision-making.

Abstract: We propose 3D-SegSync, a novel self-supervised learning (SSL) framework designed to improve segmentation accuracy for both cardiac and neurological structures. 3D-SegSync combines the state-of-the-art DINOv2 student-teacher model architecture with a 3D Vision-LSTM (xLSTM) backbone, which excels at capturing spatiotemporal dependencies and complex anatomical patterns. The SSL phase leverages large-scale unlabeled datasets to pre-train the model, while fine-tuning on labeled data ensures excellent performance across multiple imaging modalities, including CT and MRI. Our framework achieves state-of-the-art results in cardiac and brain image segmentation. 3D-SegSync sets a new benchmark for robust, modality-agnostic medical image segmentation. Code can be found here: https://github.com/Moona-Mazher/3D-SegSync SSL.

Primary Subject Area: Segmentation

Secondary Subject Area: Unsupervised Learning and Representation Learning

Paper Type: Both

Registration Requirement: Yes

Reproducibility: https://github.com/Moona-Mazher/3D-SegSync SSL

Visa & Travel: Yes

Submission Number: 77