Patient Safety Risks from AI Scribes: Signals from End-User Feedback

Jessica Dai, Anwen Huang, Catherine Nasrallah, Rhiannon Croci, Hossein Soleimani, Sarah J. Pollet, Julia Adler-Milstein, Sara G. Murray, Jinoos Yazdany, Irene Y. Chen

Keywords: End-user feedback, AI scribes, patient safety, AI monitoring

TL;DR: End-user feedback suggest the existence of patient safety issues in real-world deployments of AI Scribes

Track: Findings

Abstract: AI scribes are transforming clinical documentation at scale. However, their real-world performance remains understudied, especially regarding their impacts on patient safety. To this end, we initiate a mixed-methods study of patient safety issues raised in feedback submitted by AI scribe users (healthcare providers) in a large U.S. hospital system. Both quantitative and qualitative analysis suggest that AI scribes may induce various patient safety risks due to errors in transcription, most significantly regarding medication and treatment; however, further study is needed to contextualize the absolute degree of risk.

General Area: Applications and Practice

Specific Subject Areas: Deployment, Evaluation Methods & Validity

Data And Code Availability: No

Ethics Board Approval: Yes

Entered Conflicts: I confirm the above

Anonymity: I confirm the above

Submission Number: 87