An AI Implementation Science Study to Improve Trustworthy Data in a Large Healthcare System

Benoit Louis Marteau, Andrew Hornback, Shaun Qien Yeau Tan, Jason Woloff, Christian Lowson, May Dongmei Wang

Confirmation: I have read and agree with the IEEE BHI 2025 conference submission's policy on behalf of myself and my co-authors.

Keywords: health informatics, FHIR, OMOP-CDM, data standard, data harmonization, data quality, trustworthy AI, AI implementation science

TL;DR: Case study of Artificial Intelligence (AI) Implementation Science on large healthcare system incorporating Trustworthy AI techniques to assess data quality..

Abstract: The rapid growth of Artificial Intelligence (AI) in healthcare has sparked interest in Trustworthy AI and AI Implementation Science, both of which are essential for accelerating clinical adoption. Yet, barriers such as strict regulations, gaps between research and clinical settings, and challenges in evaluating AI systems hinder real-world implementation. This study presents an AI Implementation case study within Shriners Children’s (SC), a large multisite pediatric system, showcasing the modernization of SC’s Research Data Warehouse (RDW) to OMOP CDM v5.4 within a secure Microsoft Fabric environment. We introduce a Python-based data quality assessment tool compatible with SC’s infrastructure, an extension of OHDSI’s R/ Java-based Data Quality Dashboard (DQD) that integrates Trustworthy AI principles using the METRIC framework. This extension enhances data quality evaluation by addressing informative missingness, redundancy, timeliness, and distributional consistency. We also compare systematic and case-specific AI implementation strategies for Craniofacial Microsomia (CFM) using the FHIR standard. Our contributions include a real-world evaluation of AI implementations, integration of Trustworthy AI in data quality assessment, and evidence-based insights into hybrid implementation strategies, highlighting the need to blend systematic infrastructure with use-case-driven approaches to advance AI in healthcare.

Track: 4. Clinical Informatics

Registration Id: DRNR9S65HFR

Submission Number: 376