MediX-R1: Open Ended Medical Reinforcement Learning

ICLR 2026 Conference Submission24971 Authors

Keywords: Medical MLLMs, Reinforcement Learning, GRPO, Open-ended Reward design, Semantic evaluation, Open-ended medical reasoning

TL;DR: We introduce MediX-R1, an open-ended RL framework that equips medical multimodal LLMs with clinically grounded reasoning and evaluation for reliable free-form answers beyond multiple-choice tasks.

Abstract: We introduce MediX-R1, an open-ended reinforcement learning (RL) framework for medical multimodal large language models (MLLMs) that enables clinically grounded, free-form answers beyond multiple-choice formats. MediX-R1 fine-tunes a baseline vision–language backbone with Group Relative Policy Optimization (GRPO) and a composite reward tailored for medical reasoning: an LLM-based accuracy reward that judges semantic correctness with a strict YES/NO decision, a medical embedding–based semantic reward to capture paraphrases and terminology variants, and lightweight format and modality rewards that enforce interpretable reasoning and modality recognition. This multi-signal design provides stable, informative feedback for open-ended outputs where traditional verifiable or MCQ-only rewards fall short. To measure progress, we propose a unified evaluation framework for both text-only and image+text tasks that uses an LLM-as-judge in place of brittle string-overlap metrics, capturing semantic correctness, reasoning, and contextual alignment. Despite using only $\sim 50$K instruction examples, MediX-R1 achieves excellent results across standard medical LLM and VLM benchmarks, outperforming strong open-source baselines and delivering particularly large gains on open-ended clinical tasks (e.g., radiology summarization and report generation). Our results demonstrate that open-ended RL with comprehensive reward signals and LLM-based evaluation is a practical path toward reliable medical reasoning in multimodal models. Our trained models, curated datasets and source code will be publicly released.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 24971