ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs

ACL ARR 2026 January Submission9172 Authors

Keywords: Proactive Medical LLMs, Large Language Models, Interactive Medical Questioning, Reinforcement Learning

Abstract:

Interactive medical questioning is essential in real-world clinical consultations, where physicians must actively gather information from patients. While medical Large Language Models (LLMs) have shown impressive capabilities in static medical question answering, they predominantly operate under a reactive paradigm: generating answers directly without seeking additional information, which risks incorrect diagnoses in such interactive settings. To address this limitation, we propose ProMed, a reinforcement learning (RL) framework that transitions medical LLMs toward a proactive paradigm, equipping them with the ability to ask clinically valuable questions before decision-making. At the core of ProMed is the Shapley Information Gain (SIG) reward, which quantifies the clinical utility of each question by combining the amount of newly acquired information with its contextual importance, estimated via Shapley values. We integrate SIG into a two-stage training pipeline: (1) SIG-Guided Model Initialization uses Monte Carlo Tree Search (MCTS) to construct high-reward interaction trajectories to supervise the model, and (2) SIG-Augmented Policy Optimization, which integrates SIG and enhances RL with a novel SIG-guided Reward Distribution Mechanism that assigns higher rewards to informative questions for targeted optimization. Extensive experiments on two newly curated partial-information medical benchmarks demonstrate that ProMed significantly outperforms state-of-the-art methods by an average of 6.29% and delivers a 54.45% gain over the reactive paradigm, while also generalizing robustly to out-of-domain cases.
Our code are anonymous available at https://anonymous.4open.science/r/ProMed-1A88.

Paper Type: Long

Research Area: Clinical and Biomedical Applications

Research Area Keywords: Clinical and biomedical language models, medical question answering, clinical dialogue systems

Languages Studied: English, Chinese

Submission Number: 9172