Probabilistic Digital Twin for Data-driven Smart Weaning of Medical Circulatory Devices

Sophia Huiwen Sun, Aysin Tumay, Sonia Fereidooni, Aaron Dumas, Elise Jortberg, Rose Yu

Keywords: medical decision making, transformer, digital twin, time series forecasting, offline reinforcement learning

TL;DR: We present a probabilistic transformer-based digital twin to model MCS weaning dynamics and evaluate offline RL policies.

Abstract:

In this paper, we study the sequential decision-making for smart weaning of
mechanical circulatory (MCS) devices. MCS devices are percutaneous micro-axial
flow pumps for the treatment of cardiogenic shock patients, by providing left
ventricular unloading and forward flow of blood into the aorta. While clinical
recommendations for the weaning of MCS devices exist, the strategy varies by care
team and data-driven approaches are limited. Offline reinforcement learning (RL)
has proven to be successful in sequential decision-making tasks [8, 19], but the
prohibition of interactions with the patient as the environment constrains evaluating
RL policies. This motivates the development of probabilistic digital twin models
to simulate the environment. We propose a formulation for offline RL training
and a probabilistic Transformer-based digital twin to model the noisy circulatory
dynamics and evaluate offline RL policies. We show that our Transformer-based
digital twin (TDT) achieves 35% lower error compared to baseline models. We
also present a comprehensive benchmark on offline RL methods using TDT with
clinically relevant metrics.

Submission Number: 2