Empowering Clinicians with MeDT: A Framework for Sepsis Treatment

Aamer Abdul Rahman, Pranav Agarwal, Vincent Michalski, Rita Noumeir, Samira Kahou

Confirmation: I have read and confirm that at least one author will be attending the workshop in person if the submission is accepted

Keywords: offline reinforcement learning, clinical decision making

Abstract:

Offline reinforcement learning has shown promise for solving tasks in safety-critical settings, such as clinical decision support. Its application, however, has been limited by the need for interpretability and interactivity for clinicians. To address these challenges, we propose medical decision transformer (MeDT), a novel and versatile framework based on the goal-conditioned reinforcement learning (RL) paradigm for sepsis treatment recommendation. MeDT is based on the decision transformer architecture, and conditions the model on expected treatment outcomes, hindsight patient acuity scores, past dosages and the patient’s current and past medical state
at every timestep. This allows it to consider the complete context of a patient’s medical history, enabling more informed decision-making. By conditioning the policy’s generation of actions on user-specified goals at every timestep, MeDT enables clinician interactability while avoiding the problem of sparse rewards. Using data from the MIMIC-III dataset, we show that MeDT produces interventions that outperform
or are competitive with existing methods while enabling a more interpretable, personalized and clinician-directed approach. For future research, we release our code at https://aamer98.github.io/medical_decision_transformer/.

Submission Number: 33