Give Me The One-liner: Extracting Short Patient Summaries from Radiation Oncology Notes through Intermediates

Thibault Heintz, Suraj Pai, Marion Tonneau, Cosmin Ciausu, Shan Chen, Danielle Bitterman, Hugo Aerts, Raymond Mak

Keywords: Clinical Note Summarization, LLM, JSON, DSPy

TL;DR: This work assesses LLMs for patient one-liner generation, comparing bulk generation with the using structured intermediate output.

Track: Findings

Abstract: A patient one-liner is a very concise summary used in Radiation Oncology to streamline communication. In this study, we assess the ability of LLMs to provide apt one-liners through summarization of long-form consultation, imaging and pathology notes written by physicians for 101 patients encountered in Radiation Oncology practice. LLMs are known to struggle with long context lengths for summarization, often providing irrelevant output that does not align with the summarization intent. To tackle this, we extract one-liners via means of a two-step pipeline with an intermediate summary. We compare different methods of intermediate summarization, namely, 1) bulk summarization through structured fields (Generate once), 2) incremental summaries through structured fields and add/update operations (Chain of Key/CoK), and 3) bulk summarization through automatically optimized prompt (DSPy) using 2 open-source (deepseek-r1-8b, gemma3-27b) and one closed-source LLM (o3-mini). The intermediate summaries were passed to another automatic prompt optimization program to produce the final patient one-liner. Aggregating our observations across LLMs, we observe that CoK significantly outperformed Generate Once, demonstrating incremental summarization is more effective compared to bulk summarization when looking at structured intermediates. Secondly, Automatic prompt optimization, via DSPy, without structured fields or incremental operation outperforms Generate Once. Lastly, no significant differences were found between DSPy and CoK. Our blinded user-study found LLM generated one-liners more complete and preferred by the Radiation Oncologist compared to the human baseline. But, it was also found that they tended to produce more non-important information. Overall, our work shows the potential of automatic prompt optimization as well as structured incremental summarization to provide one-liner patient summaries that may find routine application in radiation oncology and highlights future work focused on end-to-end optimization of structured intermediates.

General Area: Applications and Practice

Specific Subject Areas: Natural Language Processing, Evaluation Methods & Validity

Data And Code Availability: Yes

Ethics Board Approval: Yes

Entered Conflicts: I confirm the above

Anonymity: I confirm the above

Code URL: https://github.com/AgentRadOnc/give_me_the_oneliner

Submission Number: 191