Back to the Future: Challenges of Sparse and Irregular Medical Image Time Series

Nico Disch, Robin Peretzke, Saikat Roy, Constantin Ulrich, David Zimmerer, Rainer Stiefelhagen, Jens Kleesiek, Klaus H. Maier-Hein

Abstract: In longitudinal medical image analysis, most work focuses on regularly sampled images, or on tasks like regression or classification. However, in the clinical context, images are frequently generated irregularly due to factors such as cost constraints. This work tackles the problem of irregularly sampled longitudinal medical imaging by evaluating methods in segmentation and reconstruction tasks. We examine the reconstruction for real-life MRI from patients with Alzheimer’s Disease (AD), where the model predicts future MRI scans from an arbitrary number of input images sampled at irregular intervals. However, experiments show that most models cannot surpass the performance of using an image from the same patient at a different time point as a baseline. Therefore, we conducted experiments on a synthetic dataset to better isolate effects in temporal learning. These experiments offer insights into model behavior and serve as a benchmark for validating models’ longitudinal learning capability, thus providing a straightforward method to assess temporal understanding in a controlled environment.