Conservative Q-Learning for Mechanical Ventilation Treatment Using Diagnose Transformer-Encoder

Yuyu Yuan, Jinsheng Shi, Jincui Yang, Chenlong Li, Yuang Cai, Baoyu Tang

Abstract: The application of artificial intelligence to mechanical ventilation has garnered significant attention, especially with the advancement of deep reinforcement learning. Mechanical ventilation is a medical procedure used in critical care to provide life support for patients with lung injuries. Physicians must continuously diagnose the patient’s condition and adjust ventilator parameters. Existing reinforcement learning treatment models provide decision support but solely focus on treatment while neglecting diagnosis. This paper proposes the DTE-CQL(Diagnose Transformer-Encoder Conservative Q-Learning) model to address this limitation. The DTE model predicts the next time-step observation and generates informative representations for auxiliary treatment. The DTE-CQL model can provide a treatment strategy and performs 1.127 times better than physicians. We trained and validated our model using the MIMIC-III dataset, demonstrating its ability to accurately predict the next time-step observation for diagnosis and provide physicians with a safe, effective, and reasonable treatment strategy.