Hybrid Student-Teacher Large Language Model Refinement for Cancer Toxicity Symptom Extraction

Reza Khanmohammadi, Ahmed I. Ghanem, Kyle Verdecchia, Ryan Hall, Mohamed Elshaikh, Bing Luo, Benjamin Movsas, Hassan Bagher-ebadian, Indrin Chetty, Tuka Alhanai, Kundan S Thind, Mohammad M. Ghassemi

Confirmation: I have read and agree with the IEEE BHI 2025 conference submission's policy on behalf of myself and my co-authors.

Keywords: Prompt Refinement, Fine-tuning, Large Language Models, Toxicity Symptom Extraction

TL;DR: This work presents a hybrid student-teacher framework using prompt refinement, retrieval-augmented generation, and fine-tuning to optimize compact language models for cancer toxicity symptom extraction, improving performance and reducing costs.

Abstract: Large Language Models (LLMs) offer significant potential for clinical symptom extraction, but their deployment in healthcare settings is constrained by privacy concerns, computational limitations, and operational costs. This study investigates the optimization of compact LLMs for cancer toxicity symptom extraction using a novel iterative refinement approach. We employ a student-teacher architecture, where the teacher model, GPT-4o, dynamically selects the most effective strategy for the student models (Zephyr-7b-beta and Phi3-mini-128) between prompt refinement, Retrieval-Augmented Generation (RAG), and fine-tuning. Our experiments on 294 clinical notes covering 12 post-radiotherapy toxicity symptoms demonstrate the effectiveness of this approach. Using 5-fold cross-validation, we observed significant improvements in F1 scores across all symptoms. The Phi3 model showed an average F1 score increase of 26%, while Zephyr achieved a 13% improvement. Notably, these enhancements were achieved at substantially lower costs, with Phi-3 being 48 times cheaper and Zephyr 30 times cheaper than GPT-4o. These results highlight the potential of iterative refinement techniques to enhance the capabilities of compact LLMs for clinical applications, offering a balance between performance, cost-effectiveness, and privacy preservation in healthcare settings.

Track: 7. General Track

Registration Id: NFN945P2NCL

Submission Number: 164