Scaling Down, Powering Up: RLHF-Enhanced Small LLMs for Healthcare Misinformation Detection

ACL ARR 2025 July Submission831 Authors

Abstract: Healthcare misinformation poses a critical threat to public well-being, necessitating detection systems that are both accurate and computationally efficient. While large language models (LLMs) have demonstrated strong performance in misinformation detection, their deployment is often constrained by high resource requirements. In this work, we investigate the effectiveness of smaller LLMs (360M–3.8B parameters) using a three-stage framework comprising standardized prompting, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF). We evaluate seven LLMs across two benchmark datasets—FakeHealth and ReCOVery—and compare them against four larger LLMs (14B–72B) and five transformer-based baselines. For the RLHF stage, we study three policy optimization methods: Binary Classifier Optimization (BCO), Contrastive Preference Optimization (CPO), and our enhanced variant, CPO**. Empirical results demonstrate that while SFT improves domain adaptation, CPO** consistently achieves the best F1 performance, enabling small LLMs to rival or even outperform significantly larger counterparts. Our findings highlight the potential of RLHF techniques to close the performance gap, offering a scalable and cost-effective solution for real-world healthcare misinformation detection.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: NLP Applications, Language Modeling, Machine Learning for NLP, Reinforcement Learning for NLP, Information Extraction, Ethics, Bias, and Fairness, Resources and Evaluation

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: English

Submission Number: 831