Failure Modes of Time Series Interpretability Algorithms for Critical Care Applications

ICLR 2025 Workshop ICBINB Submission32 Authors

Track: long paper (up to 4 pages)

Keywords: time series interpretability, critical care, deep learning, circulatory failure

TL;DR: Traditional interpretability methods have failure modes in time series data for critical care, which are tackled by learnable mask-based perturbation approaches.

Abstract:

Interpretability is a crucial aspect of deploying deep learning models in critical care, especially in constantly evolving conditions that influence patient survival. However, common interpretability algorithms face unique challenges when applied to dynamic prediction tasks, where patient trajectories evolve over time. Gradient, Occlusion, and Permutation-based methods often struggle with time-varying target dependency and temporal smoothness. This paper systematically analyzes these failure modes and supports learnable mask-based interpretability frameworks as alternatives, which can incorporate temporal continuity and label consistency constraints to learn feature importance over time. We argue that learnable mask-based approaches for dynamic time-series prediction problems provide
more reliable and consistent interpretations for applications in critical care and similar domains.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 32