PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology

Mehmet Saygin Seyfioglu

Abstract:

Diagnosing diseases through histopathology whole slide
images (WSIs) is fundamental in modern pathology but
is challenged by the gigapixel scale and complexity of
WSIs. Trained histopathologists overcome this challenge
by navigating the WSI, looking for relevant patches, taking notes, and compiling them to produce a final holistic
diagnostic. Traditional AI approaches, such as multiple instance learning and transformer-based models, fail short
of such a holistic, iterative, multi-scale diagnostic procedure, limiting their adoption in the real-world. We introduce PathFinder, a multi-modal, multi-agent framework
that emulates the decision-making process of expert pathologists. PathFinder integrates four AI agents—the Triage
Agent, Navigation Agent, Description Agent, and Diagnosis Agent—that collaboratively navigate WSIs, gather evidence, and provide comprehensive diagnoses with natural
language explanations. The Triage Agent classifies the WSI
as benign or risky; if risky, the Navigation and Description Agents iteratively focus on significant regions, generating importance maps and descriptive insights of sampled patches. Finally, the Diagnosis Agent synthesizes the
findings to determine the patient’s diagnostic classification.
Our Experiments show that PathFinder outperforms stateof-the-art methods in skin melanoma diagnosis by 8% while
offering inherent explainability through natural language
descriptions of diagnostically relevant patches. Qualitative analysis by pathologists shows that the Description
Agent’s outputs are of high quality and comparable to GPT4o. PathFinder is also the first AI-based system to surpass
the average performance of pathologists in this challenging
melanoma classification task by 9%, setting a new record
for efficient, accurate, and interpretable AI-assisted diagnostics in pathology. Data, demo, code and models are
available at https://pathfinder-dx.github.io/.