Keywords: Medical Dialogue System, Bayesian Inference, Probabilistic Reasoning, Task-Oriented Dialogue System
Abstract: Task-oriented dialogue (TOD) systems for patient history-taking improve clinical workflow efficiency by collecting key diagnostic information. Most data-driven approaches for this rely on large language models (LLMs) and mimic fast, intuitive System 1 thinking. In contrast, clinicians typically reason about potential diagnoses and use that to guide the dialog.
To bridge this gap, we propose ProbMedTOD, a TOD system that combines the conversational abilities of LLMs with the probabilistic reasoning of a disease-symptom Bayesian Network (BayesNet). At each turn, ProbMedTOD extracts information from patient utterances, updates its diagnostic hypothesis over a set of potential principal diagnoses via Bayesian inference, and generates the next question using a supervised policy LLM trained on dialogue data.
The BayesNet structure is programmatically constructed from clinical documents, while its parameters are inferred automatically via self-consistent prompting of an LLM, removing the need for expert-labeled data.
We develop a patient simulator that uses patient profiles informed by the dialogue context and engages in realistic end-to-end interactions with the system, enabling evaluation of dialogue-level success. ProbMedTOD significantly outperforms LLM and retrieval-based baseline in next-question prediction and dialogue-level success, obtaining ~20 pt MRR improvement in simulation experiments.
Supplementary Material: zip
Primary Area: other topics in machine learning (i.e., none of the above)
Submission Number: 18682
Loading