Reinforcement Learning for Blood Glucose Control: Challenges and Opportunities

Ian Fox, Jenna Wiens

Keywords: Deep Reinforcement Learning, Diabetes, Blood Glucose Control, Artificial Pancreas

Abstract: Individuals with type 1 diabetes (T1D) lack the ability to produce the insulin their bodies need. As a result, they must continually make decisions about how much insulin to self-administer in order to adequately control their blood glucose levels. Longitudinal data streams captured from wearables, like continuous glucose monitors, can help these individuals manage their health, but currently the majority of the decision burden remains on the user. To relieve this burden, researchers are working on closed-loop solutions that combine a continuous glucose monitor and an insulin pump with a control algorithm in an `artificial pancreas.' Such systems aim to estimate and deliver the appropriate amount of insulin. Here, we investigate the utility of reinforcement learning (RL) techniques for automated blood glucose control. Through a series of experiments, we compare the performance of different deep RL approaches to non-RL approaches. We find that the RL approaches are competitive with the baselines (achieving an average risk across three patients of 8.56 vs. the baseline 8.48) and are better able to handle latent behavioral patterns (improving risk in one patient to 9.26 vs. the baseline 11.80). These preliminary results suggest that RL could be useful for improving blood glucose control algorithms.