REFLEX-Med: Reinforcement for Label-Free Explainability in Unified Medical Reasoning

ICLR 2026 Conference Submission4734 Authors

Keywords: medical reasoning, large vision-language models, explainability

Abstract: Clinicians urgently need explanations they can audit, not merely fluent chains. Yet prevailing practices conflate interpretability with subjective human/LLM rationales, with post-hoc visuals loosely aligned to answers, or with answer rationale consistency. These proxies are annotation-hungry, bias-prone, and crucially do not certify process verifiability: where the model looked and why it looked there. Meanwhile, reinforcement learning from feedback excels at answer verifiability but offers little support for constraining the provenance of attention or penalizing visually ungrounded reasoning. We introduce REFLEX-Med, a reinforcement framework that instantiates label-free explainability through two verifiable prerequisites: (i) faithful visual grounding that is text-conditioned localization in the image, and (ii) bi-directional cross-modal provenance, that is a cycle of mutual traceability across image-text and text-text semantics. REFLEX-Med couples curriculum GRPO with two frozen rewards computed by a medical vision-language encoder: a visual fidelity reward aligning text-conditioned saliency between the model's own answer and an anchor text, and a bi-modal provenance reward enforcing image-text and text-text consistency in embedding space. Together with standard format and semantic-matching rewards, REFLEX-Med resists large VLM hallucination and attention-think drift, improving both answer quality and auditable faithfulness on unified medical reasoning (open and close-ended VQA) all without human or LLM rationale annotations.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 4734