When LLMs Decide Who Gets Care: A Vision for Multi-Agent Systems in High Stakes Clinical Decision-Making

Divya Sharma, Ghazal Azarfar, Bima J. Hasjim, Mamatha Bhat

Keywords: Multi-Agent Systems, Clinical Decision-Making, Liver Transplant, Fairness in AI, Explainability, Large Language Models (LLMs)

TL;DR: Simulating multi-agent clinical decision-making with LLMs, we show how AI can reinforce societal inequities in transplant access and propose a framework for building fairer, explainable healthcare systems.

Abstract: As large language models (LLMs) transition from static predictors to autonomous agents, a promising application lies in simulating real-world, multi-disciplinary clinical committees responsible for life-or-death decisions such as organ transplant eligibility. This vision paper explores the design and deployment of multi-agent LLM systems that emulate role-specialized clinicians collaborating to assess high-stakes medical cases. Using empirical insights from a simulation of a liver transplant selection committee, we highlight how even highly accurate agents can propagate disparities in the absence of subgroup-sensitive reasoning and explainability. We argue that multi-agent LLMs must go beyond role emulation to incorporate counterfactual rationalization, inter-agent transparency, and clinician-in-the-loop arbitration. Our vision sets forth a roadmap for building accountable, equitable, and trustworthy multi-agent LLM systems that can support, not replace critical clinical deliberation.

Submission Number: 21