Informative Irregularity as a Diagnostic for Model Robustness

Tamara Krafft, Bernhard Bauer

Keywords: Machine Learning, Health Time Series, Irregularity, Data Quality, Emergency Departement, Early Warning, Irregular Data, Sparse Data

TL;DR: Extracting the timing of EHR measurements as features and stratifying patients based on irregularity, we show that informative irregularity serves as an independent signal that boosts performance by up to 8.0% AUPRC in volatile clinical workflows.

Abstract:

Electronic health records frequently contain irregularly sampled data. The specific timing of clinical observations itself can contain informative signals regarding patient acuity. To explicitly model this behavior, we extract structural workflow metadata from 3,497 MIMIC-IV-ED stays and introduce a stratification approach to evaluate model robustness across distinct irregularity regimes.
Integrating structural metadata yields a $+4.1\%$ AUPRC improvement over the native baseline. Furthermore, feature importance analysis reveals that structural metadata provides a statistically independent diagnostic signal over vital sign data (mean correlation $|\rho| = 0.041$ between top structural and physiological features).
Crucially, stratification reveals that the utility of this metadata peaks when sampling does not adhere to local routine rhythms, yielding an $+8.0\%$ AUPRC improvement over the native baseline. These findings demonstrate that quantifying distinct irregularity manifestations provides a diagnostic framework to map how model performance fluctuates across different clinical workflows.

Submission Number: 51