DocCHA: Towards LLM-Augmented Interactive Online diagnosis System
Abstract: Large Language Models (LLMs) have demonstrated great capabilities in addressing many application tasks. Among various applications, one eye-catching domain is Conversational Health Agents (CHAs), which are interactive conversational systems that can provide people with various health-related services. However, existing CHAs mainly focus on providing static health services, and lack interactive online diagnosis for patients. In the clinical setting, the initial symptoms that patients provide may lack comprehensiveness and detail, thus online interaction with patients to request additional information is important. To alleviate this problem, we propose DocCHA, an online interactive diagnosis system that interacts with patients by requesting additional information and continuesly improving the diagnosis confidence until providing patients with a reliable diagnosis. Moreover, DocCHA leverages Retrieval Augmented Generation (RAG) with Google search API, StatPearls and Wikipedia to provide patients with detailed and reliable health suggestions. We evaluate DocCHA's performance on the IMCS21 dataset, a Chinese online diagnosis dataset consists of conversations between patients and doctors. Experimental results show that DocCHA's diagnosis accuracy reaches 89.2\% with 4 rounds of additional information request interactions with patients. Besides, the generated suggestion after RAG outperforms the direct prompt in terms of relevance, coherence, accuracy and completeness.
Paper Type: Long
Research Area: Dialogue and Interactive Systems
Research Area Keywords: Dialogue and Interactive Systems, NLP Applications
Contribution Types: NLP engineering experiment
Languages Studied: English
Submission Number: 1419
Loading