A Framework for Fine-Grained Complexity Control in Health Answer Generation

Daniel Jorge Bernardo Ferreira, Tiago Almeida, Sérgio Matos

Keywords: Health literacy, Medical text simplification, Readability assessment, Text complexity, Controlled text generation

TL;DR: A framework for generating health answers at adjustable complexity levels using a specialized medical text formula and a model fine-tuned on 21 readability levels to match diverse health literacy needs.

Abstract:

Effective communication of health information requires adapting complexity to match the target audience's literacy level. However, manually simplifying medical content is both time-consuming and difficult to scale. To address this challenge, we developed a new framework for automatically generating health answers at multiple complexity levels.

We began by collecting 166 linguistic features to quantify text complexity, including traditional readability metrics (e.g., Flesch-Kincaid, SMOG), medical terminology usage (e.g., UMLS coverage, medical entity recognition), syntactic complexity, semantic coherence, and LLM-based measures (e.g., masked language modeling, LLM-as-a-judge). Applying these features to a custom dataset of parallel health texts and external medical benchmarks, we used feature selection to identify 13 key metrics that reliably distinguish between simple and complex text pairs. From these, we derived a complexity scoring formula by combining the metrics with weights learned from a logistic regression model.

Using this formula, we created a large multi-level dataset of health question-answer pairs, ranging from elementary patient-friendly explanations to advanced technical summaries.
The initial QA pairs came from established datasets including LiveQA, MedicationQA, and MEDIQA-AnS. We then used LLaMA-based language models with carefully engineered prompts to transform the original answers into five different versions ordered by complexity.
Finally, we fine-tuned a large language model on this dataset, incorporating special tokens to control the complexity of the generated text. The resulting model can generate health answers at fine-grained complexity levels, allowing users to select the desired level of detail and technicality.

Archival Status: Archival

Paper Length: Long Paper (up to 8 pages of content)

Submission Number: 326