Generative AI in the Hospital: A Participatory Assessment of Healthcare Needs and Challenges

Giorgia Carra, Bogdan Kulynych, François Bastardot, Noémie Boillat-Blanco, Jean Louis Raisaro

Keywords: GenAI for healthcare, Medical LLMs, Technology readiness level, Participatory approach, Feasibility study

TL;DR: This study uses a participatory approach to identify healthcare needs for LLMs, followed by a feasibility analysis considering technical, ethical, and regulatory challenges.

Abstract:

Although Large Language Models (LLMs) have shown promising performance in several medical applications, their deployment in the medical domain poses unique challenges of ethical, regulatory, and technical nature. In this study, we employ a systematic participatory approach to investigate the needs and expectations regarding clinical applications of LLMs at a major European university hospital. Having identified potential LLMs use-cases in collaboration with stakeholders, we assess the current feasibility of these use-cases focusing on pressing challenges such as regulatory frameworks, data protection regulation, bias, hallucinations, and deployment constraints.
This work provides a framework for a participatory approach to identifying institutional needs with respect to introducing advanced technologies into clinical practice, and a realistic analysis of the technology readiness level of LLMs for medical applications, highlighting the issues that need to be overcome to enable safe, ethical and regulatory compliant use of medical LLMs.

Submission Number: 32