Reinforcement Learning-based Decision-making for Renal Replacement Therapy in ICU-acquired AKI Patients

Haoyuan Zhang, Minqi Xiong, Tongyue Shi, Wentie Liu, Haowei Xu, Huiying Zhao, Guilan Kong

Keywords: Reinforcement Learning, Acute Kidney Injury, Renal Replacement Therapy, Renal Function Recovery, Decision Support Tool

TL;DR: We develop a value-based reinforcement learning decision-support model that recommends personalized RRT treatment plans for AKI patients using data from EHRs..

Abstract: In the Intensive Care Unit (ICU), Renal Replacement Therapy (RRT) serves as an effective tool for improving fluid balance and promoting renal function recovery in patients with severe Acute Kidney Injury (AKI). The need for RRT, the timing of its initiation and discontinuation, and the selection of its modalities require careful consideration due to its potential impact on renal function recovery and the associated risks. However, the existing Kidney Disease Improving Global Outcomes (KDIGO) guidelines provide only general recommendations, leaving room for physicians to provide subjective judgment, and thus a personalized RRT decision-making support tool is in urgent need. This study proposed to employ a value-based reinforcement learning approach to model the relationships between patient states, RRT decisions, and renal function recovery. This approach allows for action recommendations under various patient states, and can balance both short- and long-term patient outcomes. In the modelling process, patients' sequential state data was utilized, three RRT-related strategies were considered, and the reward function was defined based on the rate of estimated glomerular filtration rate (eGFR) change. The proposed reinforcement learning-based RRT decision-making model was experimented on the AKI dataset extracted from the publicly available ICU dataset MIMIC-IV. The experimental results showed that the RRT strategy recommendations provided by our developed reinforcement learning-based decision support tool were consistent with clinical guideline and some recommendations are more rational than actual actions in specific patient cases.

Submission Number: 31