Budget-Aware Routing for Long Clinical Text

ACL ARR 2026 January Submission10229 Authors

Keywords: Long-document processing, Text summarization, Budget-constrained inference, Context selection / context construction, Clinical NLP, Evaluation of large language models

Abstract:

A growing consideration for large language models is the token cost per query and the cost of deployment.
In clinical settings, the input is long, heterogeneous, and redundant, while the downstream task is often short and high stakes.
This paper studies a concrete primitive for long context use: select a subset of document units under a strict token budget so that an off-the-shelf generator can operate within fixed cost and latency.
We formulate budgeted context selection as a knapsack-constrained subset selection problem with two design axes: \emph{unitization} (how to segment documents) and \emph{selection} (which units to keep).
For selection, we present RCD, a monotone submodular objective coupling relevance, coverage, and diversity.
For unitization, we evaluate sentence-level, section-based, window-based, and cluster-based strategies.
We introduce a routing heuristic that chooses among methods based on budget regime.
Across MIMIC discharge notes, Cochrane abstracts, and L-Eval, we find that optimal strategy depends on evaluation paradigm: for extractive evaluation, positional heuristics dominate at low budgets; for LLM-based generation, diversity-aware selection (MMR) provides consistent gains.
Unitization matters less than selector choice--cluster-based grouping hurts performance, while other strategies perform similarly.
We observe that ROUGE saturates for LLM-generated summaries and that BERTScore better captures quality differences between methods.

Paper Type: Long

Research Area: Clinical and Biomedical Applications

Research Area Keywords: Clinical and Biomedical Applications, Efficient/Low-Resource Methods for NLP, Ethics, Bias, and Fairness, Generation

Contribution Types: Approaches to low-resource settings, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models, Data resources

Languages Studied: English

Submission Number: 10229