BTReport: A Framework for Brain Tumor Radiology Report Generation with Clinically Relevant Features

Juampablo E Heras Rivera, Dickson T Chen, Tianyi Ren, Daniel Low, Asma Ben Abacha, Alberto Santamaria-Pang, Mehmet Kurt

Keywords: Brain MRI, Radiology report generation, VASARI, Midline shift, Open dataset, Multimodal learning, Neurooncology

TL;DR: We generate Findings sections for brain tumor imaging using a two-stage approach: first quantitative features are extracted, then LLMs are used for report writing.

Abstract: Recent advances in radiology report generation (RRG) have been driven by large paired image-text datasets; however, progress in neuro-oncology RRG has been limited due to a lack of open paired image-report datasets. Here, we introduce BTReport, an open-source framework for brain tumor RRG that constructs natural language radiology reports using reliably extracted quantitative imaging features. Unlike existing approaches that rely on large general-purpose or fine-tuned vision-language models for both image interpretation and report composition, BTReport first performs deterministic feature extraction of clinically-relevant features, then uses large language models only for syntactic structuring and narrative formatting. By separating RRG into a deterministic feature extraction step and a report generation step, the generated reports are completely interpretable and less prone to hallucinations. We show that the features used for report generation are predictive of key clinical outcomes, including survival time, and reports generated by BTReport are more closely aligned with reference clinical reports than existing baselines for RRG. Finally, to further research in neuro-oncology RRG, we introduce BTReport-BraTS, a companion dataset that augments BraTS imaging with synthetically generated radiology reports produced with BTReport. Code for this project can be found at: \url{https://github.com/KurtLabUW/BTReport}.

Primary Subject Area: Integration of Imaging and Clinical Data

Secondary Subject Area: Generative Models

Registration Requirement: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 115