Towards a Real-time Clinical Agenda Setting System for Enhancing Clinical Interactions in Primary Care Visits

Kuk Jin Jang, Sameer Bhatti, Sydney Pugh, Chimezie Maduno, Sarang Sridhar, Sriharsha Mopidevi, Eric Eaton, Kevin Johnson

Keywords: patient-provider communication, agenda-setting, real-time systems, large language models, real-time summarization

Abstract: Technology has increasingly hindered meaningful engagement between patient and providers during primary care visits, often detracting from effective communication. However, artificial intelligence (AI) advancements present new opportunities to enhance and improve patient-provider communication. A promising application is the use of AI to identify and highlight agenda items for discussion during visits and to summarize relevant clinical details in real-time. This study explores the feasibility, potential, and challenges of developing a real-time automated agenda-setting system leveraging generative AI, specifically large language models (LLMs). From a dataset of recorded and annotated simulation visits, we evaluate the performance of LLMs in identifying agenda items and capturing associated clinical details within the conversation flow. In particular, we focus on the impact of realtime constraints and contextual factors on the ability to detect and summarize relevant items. Our findings suggest that optimizing performance requires a balance between providing contextual information through both summaries and the actual conversation. Based on these results, we discuss the challenges involved in developing a real-time agenda-setting system and offer recommendations for future advancements.

Submission Number: 71