Natural Language Grounded Reinforcement Learning for Clinical Decision-Making in Virtual Patient Simulations

Niyel Hassan, Benjamin Liu, Jason Tsai, Jeffrey K Jopling, Dana Lin, Edward Melcer, Cara Liebert

Keywords: Reinforcement learning, natural language state representations, sequential clinical diagnosis, clinical decision making, virtual patient simulations, constrained action spaces, healthcare AI

Abstract: We present a reinforcement learning framework for training agents in simulated clinical diagnostic tasks within virtual patient simulations. Each patient case is cast as a Markov Decision Process with a hybrid state that fuses semantic encodings of clinical text with structured physiology and a masked Proximal Policy Optimization policy that enforces clinical action feasibility. The learned policy is stable and competent, achieving a recall of 0.75 for clinically indicated actions while avoiding over 96% of harmful actions. Domain-specific language encoders materially improve performance, underscoring the value of a language-grounded state. Crucially, we find that a conservative checklist strategy, which favors completeness over efficiency, reveals disparities across specialties and demographics, including a safety drop in geriatric cases. Our framework offers a rigorous testbed and strong baseline for language-based clinical policy learning and clarifies targets for improving efficiency, generalization, and fairness in reinforcement learning agents for clinical decision-making.

Archival Option: The authors of this submission want it to appear in the archival proceedings.

Submission Number: 119