Is classification all you need for radiology report generation?
Keywords: radiology report generation, large language model, multi-modalities
Abstract: Automatic radiology report generation is an advanced medical assistive technology capable of producing coherent reports based on medical images, akin to a radiologist. However, current generative methods exhibit a notable gap in clinical metrics when compared to medical image classification. Recently, leveraging diagnostic results to improve report quality has emerged as a promising approach. We are curious whether training a classifier that encompasses all possible long-tailed and rare diseases could enhance the robustness of reports. To investigate this question, this study designs an evaluation framework that integrates long-tail scenarios and summarizes potential combinations of LLM-based report generation models. We assess the impact of classification on report quality across four benchmarks. Initially, we introduce LLM-based language and clinical metrics and develop a pipeline to evaluate the model's performance on both in-domain and out-of-distribution (OOD) long-tail scenarios. Subsequently, we conduct a systematic evaluation of all potential model combinations. Our findings reveal that: 1) the impact of classification on report quality is positively correlated with the performance of classifiers, but the gap still exists, and 2) while classification can enhance report quality in in-domain long-tail scenarios, its benefits for OOD scenarios are limited.
Primary Area: other topics in machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2875
Loading