Examining the Vulnerability of Multi-Agent Medical Systems to Human Interventions for Clinical Reasoning

Benjamin Liu, Dillon Mehta, Rishi Malhotra, Adam Zobian, Yong Ying Tan, Samir Chopra, Daniella Rand, Natalie Pang, Abhiram Gudimella, Raghav Thallapragada, Derek Jiu, Kevin Zhu

Keywords: Human interventions, Auditabiliy, Clinical reasoning, Multi-Agent, Regulatory oversight

TL;DR: Human interventions at vulnerable points in multi-agent medical AI can meaningfully alter the diagnostic trajectory, improving accuracy when correct but destabilizing reasoning and amplifying bias when incorrect.

Abstract: Human interventions at fault points can alter the diagnostic accuracy of multi-agent medical systems. We defined fault points as moments in doctor-patient conversations, where the Doctor Agent's reasoning became most vulnerable to external influence and change. Using a MedQA dataset, this study analyzed simulated doctor-patient conversations to measure how fault point interventions shifted reasoning and accuracy. Correct intervention methods showed an improvement in baseline diagnostic accuracy up to 40\%, while incorrect or bias-related interventions degraded performance by up to 6\%, and increased diagnostic drift and uncertainty. Beyond accuracy, the analysis revealed behavioral patterns between cognitive biases in simulated Medical AI and real-world clinical practice. Examples included premature closure and susceptibility to misleading cues, which are concerning in healthcare, where reliability and fairness are critical. This makes fault points natural audit checkpoints for oversight or human verification. Overall, the findings reveal that priming large language models (LLMs) at fault points can improve reliability, expose drift and bias, and support stress-testing for certification.

Submission Number: 77