MEDVISTAGYM: A Scalable Training Environment for Thinking with Medical Images via Tool-Integrated Reinforcement Learning

ACL ARR 2026 January Submission2991 Authors

Keywords: Medical Reasoning, VLM, Agentic Training

Abstract:

Vision language models (VLMs) achieve strong performance on general image understanding but struggle to "think with medical images", especially when performing multi-step reasoning through iterative visual interaction. Medical VLMs often rely on static visual embeddings and single-pass inference, preventing models from re-examining, verifying, or refining visual evidence during reasoning.
While tool-integrated reasoning offers a promising path forward, open-source VLMs lack the training infrastructure to learn effective tool selection, invocation, and coordination in multi-modal medical reasoning. We introduce MedVistaGym, a scalable and interactive training environment that incentivizes tool-integrated visual reasoning for medical image analysis.
MedVistaGym equips VLMs to determine when and which tools to invoke, localize task-relevant image regions, and integrate single or multiple sub-image evidence into interleaved multimodal reasoning within a unified, executable interface for agentic training.
Using MedVistaGym, we train MedVistaGym-R1 to interleave tool use with agentic reasoning through trajectory sampling and end-to-end reinforcement learning. Across six medical VQA benchmarks, MedVistaGym-R1-8B exceeds comparably sized tool-augmented baselines by 19.10% to 24.21%, demonstrating that structured agentic training—not tool access alone—unlocks effective tool-integrated reasoning for medical image analysis.

Paper Type: Long

Research Area: Clinical and Biomedical Applications

Research Area Keywords: vision question answering

Contribution Types: Model analysis & interpretability, Data analysis

Languages Studied: English

Submission Number: 2991