FetalExtract-LLM: Structured Information Extraction from Free-Text Fetal MRI Reports Based on Privacy-Ensuring Open-Weights Large Language Models

Mingxuan Liu, Yijin Li, Juncheng Zhu, Hongjia Yang, Yiming Huang, Haoxiang Li, Yifei Chen, Xuguang Bai, Yi Liao, Haibo Qu, Qiyuan Tian

Abstract: Magnetic resonance imaging (MRI) is essential for evaluating fetal abnormality, providing superior soft tissue contrast to ultrasound. Radiologists write unstructured free-text reports documenting findings in MRI data, but this format hinders retrospective access and secondary data usage. Structuring these reports facilitates cohort identification for research, supports the development of medical AI systems, and enables educational applications. However, the diverse linguistic styles and inconsistent formatting of free-text reports pose significant challenges for structured information extraction. Despite the capabilities of proprietary large language models (LLMs) like GPT, Claude and Gemini in automating extraction via zero-shot prompting, their cost and privacy limitations prohibit their use. To address these problems, this study introduces FetalExtract-LLM, the first fetal MRI report structured information extraction model developed through instruction tuning of an open-weights, privacy-preserving LLM. Experimental results show that FetalExtract-LLM achieves 0.987 average F1-score in per-key matching, on par with proprietary models, with additional metrics of 1.000 JSONable accuracy, 0.966 domain compliance accuracy, 0.829 positive finding accuracy, and 0.770 exact matching accuracy.

External IDs:dblp:conf/miccai/LiuLZYHLCBLQT25