Image-To-Tree with Recursive Prompting

James Batten, Matthew Sinclair, Ying Bai, Ben Glocker, Michiel Schaap

Abstract: Extracting complex structures from grid-based data is a common yet challenging key step in automated medical image analysis. A conventional method for recovering tree-structured geometries involves calculating the minimal cost path via intermediate representations derived from segmentation masks. This method faces limitations in projective imaging of 3D tree-structured data, such as coronary arteries, due to overlapping branches in the 2D view. In this study, we introduce a new method for predicting tree connectivity, reframing it as an optimization problem in a recursive process. We propose a two-stage model leveraging the UNet and Transformer architectures, combined with an image-based prompting technique. The method yields compelling results on synthetic and simulated data with real geometry, surpassing a shortest-path baseline.