Towards Human-AI Collaboration in Healthcare: Guided Deferral Systems with Large Language Models

Joshua Strong, Qianhui Men, Alison Noble

Keywords: Large Language Models, Human-AI Collaboration, Healthcare, Medical Report Parsing

TL;DR: We develop a deferral system using large language models of which is able to parse medical reports for clinical classifications and defer uncertain cases to humans with textual guidance

Abstract: Large language models (LLMs) present a valuable technology for various applications in healthcare, but their tendency to hallucinate introduces unacceptable uncertainty in critical decision-making situations. Human-AI collaboration (HAIC) can mitigate this uncertainty by combining human and AI strengths for better outcomes. This paper presents a novel *guided deferral* system that provides intelligent guidance when AI defers cases to human decision-makers. We leverage LLMs' verbalisation capabilities and internal states to create this system, demonstrating that fine-tuning small-scale LLMs with data from large-scale LLMs greatly enhances performance while maintaining computational efficiency and data privacy. A pilot study showcases the effectiveness of our proposed deferral system.

Submission Number: 26