Attention-to-Survival: Multimodal Fracture Risk Prediction Based on Pelvic Radiographs and Clinical Data from the Study of Osteoporotic Fractures

Niklas Christoph Koser, Marten Johannes Finck, Silja Janßen, Coenraad Mouton, Li-Yung Lui, Steven Ron Cummings, Kevin Köser, Soren Pirk, Jan-Bernd Hövener, Claus-C. Glüer

Keywords: Survival Analysis, Hip Fracture Risk, Multimodal, Deep Learning

TL;DR: We introduce a multimodal deep survival model that combines pelvic radiographs and clinical data to predict individualized hip fracture risk more accurately than BMD or FRAX.

Abstract: Osteoporotic changes in the hip structure render the proximal femur particularly vulnerable to fractures, which leads to severe consequences for patients' health and significant socioeconomic burdens, a strongly increasing problem in aging populations. Accurate risk estimation is therefore essential for initiating timely preventive measures. However, the current clinical standard measures bone mineral density (BMD) and the Fracture Risk Assessment Tool (FRAX) provide only limited predictive value. Neither BMD nor FRAX capture structural characteristics that could be derived from pelvic radiographs that are widely available. To address this gap, we present the Attention-to-Survival Fusion (ATSF) model, a multimodal survival analysis framework that combines clinical risk factors (CRFs) with pelvic radiograph features. An attention-based architecture equipped with a deep conditional transformation model (DCTM) prediction head enables accurate estimation of time-dependent fracture risk. The ATSF model is designed to accommodate missing clinical variables, handle all forms of non-informative censoring, and provides modality-specific interpretability through the attention mechanisms. It was developed, validated and tested with data of 7825 women from the Study of Osteoporotic Fractures (SOF) followed for fracture incidence for 23 years. We benchmark ATSF against established baselines, including FRAX, the Cox proportional hazards model (CoxPH), and a deep learning reference model. Our results demonstrate significant superior performance across concordance index (C-index) and area under the receiver operating characteristic curve (AUC), indicating the importance of integrating radiographic and clinical data within a unified survival framework. Furthermore, offering improved interpretability and a scalable multimodal design, the proposed method provides a promising alternative for advancing individualized hip-fracture risk prediction in osteoporosis research and precision medicine.

Primary Subject Area: Integration of Imaging and Clinical Data

Secondary Subject Area: Application: Radiology

Registration Requirement: Yes

Reproducibility: https://github.com/nkoser/Attention-to-Survival

Visa & Travel: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 231