Benchmarking the Reproducibility of Brain MRI Segmentation Across Scanners and Time

Ekaterina Kondrateva

Keywords: Machine Learning, Brain Morphometry, MRI, Multi-Scanner Variability, Dice, FreeSurfer, SynthSeg, Segmentation, Statistics, Test Retes, Domain Shift

TL;DR: Cross-scanner, longitudinal reproducibility of brain MRI morphometry: FastSurfer + FreeSurfer v8 (SynthSeg) on SIMON (17y) & SRPBS (9-site). Small subcorticals vary ~7–10%; Surface-Dice QC keeps >94%; interpolation ~1.7% vs ~0.07% template.

Abstract:

Accurate and reproducible brain morphometry from structural MRI is critical for monitoring neuroanatomical changes across time and imaging domains. Although deep learning has accelerated segmentation workflows, scanner-induced variability and reproducibility limitations remain—particularly in longitudinal and multi-site settings. In this study, we benchmark two state-of-the-art pipelines—\textit{FastSurfer} and \textit{SynthSeg}—both integrated into \textit{FreeSurfer}, one of the most widely adopted tools in neuroimaging. Using two complementary datasets—a 17-year single-subject longitudinal cohort (SIMON) and a 9-site test-retest cohort (SRPBS)—we quantify inter-scan segmentation variability using Dice, Surface Dice, Hausdorff Distance (HD95), and Mean Absolute Percentage Error (MAPE).
Our results reveal up to 7–8\% volume variation in small subcortical structures such as the amygdala and ventral diencephalon, even under controlled test-retest conditions. This raises a critical question: is it feasible to detect subtle longitudinal changes—on the order of 5–10\%—in pea-sized brain regions, given the magnitude of domain-induced morphometric noise? We further analyze the effects of registration choices and interpolation modes, and propose surface-based quality filtering to improve reliability. This work provides a reproducible benchmark and calls for harmonization strategies to enable robust morphometry in real-world neuroimaging studies.
%
Our code is available at~\url{https://github.com/kondratevakate/brain-mri-segmentation}.}

Submission Number: 21