Segmentation-Informed Landmark Regression for Robust Localization in Fluoroscopic Imaging

David Jozef Hresko, Peter Drotar

Keywords: landmark detection, semantic segmentation, medical registration, medical imaging

Abstract: Accurate and robust localization of anatomical landmarks in fluoroscopic images is essential for image-guided interventions, yet remains challenging due to low contrast, noise, and overlapping anatomical structures. In this work, we propose a framework that jointly performs bone structure segmentation and landmark heatmap prediction to improve landmark localization accuracy. Our method is built on an HRNet-based architecture augmented with a custom decoder, enabling high-resolution feature preservation while learning rich spatial context for both tasks. To guide the network toward anatomically consistent predictions, we introduce a composite loss function that integrates dice-sørensen coefficient combined with cross-entropy for segmentation quality, Kullback–Leibler divergence loss for heatmap regression, and a novel anatomical consistency loss that penalizes landmark predictions falling outside their corresponding segmented bone regions. Experiments demonstrate that coupling segmentation with landmark regression significantly improves localization robustness compared to other approaches, resulting in a mean localization error of just 10.25 pixels. The proposed approach provides a reliable foundation for downstream tasks in radiology.

Primary Subject Area: Image Registration

Secondary Subject Area: Segmentation

Registration Requirement: Yes

Visa & Travel: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 230