The Power of Data: How LSTMs Outshine Disease Progression Modeling with Two Simple Mechanisms

Seongsoo Kim, Min Chi

Keywords: LSTM, Septic Shock, Disease Progression Modeling, Time-Aware, VRNN, Transformer, Bi-Directional

TL;DR: Incorporating time-awareness and bidirectionality enhances LSTMs, VRNNs, and Transformers for disease progression modeling, especially in predicting septic shock across large healthcare datasets.

Abstract:

Much of prior efforts have focused on Disease Progression Modeling (DPM) using Electronic Health Records (EHRs). EHRs, however, present significant challenges for deep learning models such as Long Short-Term Memory (LSTM), Variational Recurrent Neural Networks (VRNN), and Transformer due to the inherent complexities and variabilities within the data. Effectively addressing these variabilities is crucial for improving the performance and interpretability of such models. In this work, we propose two mechanisms to tackle key variabilities in EHR data: a "bi-directional" mechanism to account for the need to infer the underlying physical state in both forward and backward directions, and a "time-aware" mechanism to address irregular time intervals between consecutive events. We theoretically validate and empirically evaluate the impact of these two mechanisms across three state-of-the-art deep learning models in three distinct healthcare systems. Our results showed that the influence of the two mechanisms—bidirectionality and time-awareness—surpasses the differences between specific deep learning models. Across all three models, the performance hierarchy consistently follows: Bidirectional & Time-Aware > Time-Aware > Bidirectional > Original model, across all three healthcare systems. Notably, the Bidirectional Time-Aware LSTM matches or exceeds the performance of the corresponding VRNN and Transformer models in every system tested.

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10663