Reliable Evaluation of MRI Motion Correction: Dataset and Insights

Kun Wang, Tobit Klug, Stefan Ruschke, Jan Kirschke, Reinhard Heckel

Keywords: 3D MRI motion correction, Accelerated MRI, Dataset, Evaluation approach, Image reconstruction

Abstract:

Correcting motion artifacts in scientific and medical imaging is important, as they significantly impact image quality.
However, evaluating deep learning-based and classical motion correction methods remains fundamentally difficult due to the lack of accessible ground-truth target data.
To address this challenge, we study three evaluation approaches: real-world evaluation based on reference scans, simulated motion, and reference-free evaluation, each with its merits and shortcomings.
To enable evaluation with real-world motion artifacts, we release PMoC3D, a dataset consisting of unprocessed $\textbf{P}$aired $\textbf{Mo}$tion-$\textbf{C}$orrupted $\textbf{3D}$ brain MRI data.
To advance evaluation quality, we introduce MoMRISim, a feature-space metric trained for evaluating motion reconstructions.
We assess each evaluation approach and find real-world evaluation together with MoMRISim, while not perfect, to be most reliable.
Evaluation based on simulated motion systematically exaggerates algorithm performance, and reference-free evaluation overrates oversmoothed deep learning outputs.

Primary Area: datasets and benchmarks

Submission Number: 24038