Learning from Missing Values: Encoding Missingness in Representation-Space for LSTM Time Series Forecasting

TMLR Paper7450 Authors

Abstract: While many state-of-the-art techniques reconstruct incomplete time series datasets by replacing gaps with modeled estimates, we propose an alternative: encode missing values as an extremal sentinel value, allowing a prediction model to learn from the pattern of missingness. Incomplete data is a common problem in real-world time series forecasting, particularly in environmental monitoring where sensor failures can cause continuous gaps in data. This paper proposes the Min-Std method, a novel computationally efficient imputation strategy that encodes missingness in representation-space with an extremal statistical sentinel $(min - \sigma)$ mapped to $0$ under Min-Max scaling. The result is that instead of training a model on (possibly imprecise) estimates for missing data, we simply replace the missing value with a sentinel the model can recognize to mean `uninformative'. By ensuring this sentinel is uniquely mapped to $0$, the only $0$ values the model will receive are either missing values, or values dropped by a dropout regularizer. We compare prediction results using our Min-Std imputation strategy against 12 imputation methods (including Kalman Smoothing and MissForest) across 6 different transformations on 28 distinct environmental datasets. Friedman's nonparametric test and critical difference ranking demonstrate that Min-Std imputation consistently yields superior predicting performance (measured by KGE, NMSE, and F1 Score) compared to complex model-based alternatives while being orders of magnitude faster (e.g. $0.02$s vs $500$s$+$). Our findings suggest that single-channel, explicit representation-space encodings of missingness are preferable to reconstruction-based imputation.

Submission Type: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=Yhjs9lXQAo

Changes Since Last Submission: Following the desk-reject note from the previous submission feedback of 'modified template', I audited the document preamble and found that I had inadvertently modified document spacing. I have removed/edited all preamble that could have affected the formatting incorrectly, and ensured that it aligns with the example main.tex file provided by TMLR. These formatting changes have slightly reduced the page count by approximately 1/2 of a page, from 12 pages to ~11.5 pages.

Assigned Action Editor: ~Yingnian_Wu1

Submission Number: 7450