Variability Aware Recursive Neural Network (VARNN): A Residual-Memory Model for Capturing Temporal Deviation in Sequence Regression Modeling

Haroon Gharwi, Kai Shu

Keywords: Time-series regression, non-stationarity, residual learning, recurrent neural networks, distribution shift, system identification.

TL;DR: We propose a VARNN, a residual-memory recursive neural network that improves robustness to variability in time-series regression.

Abstract: Real-world time series data exhibit nonstationary behavior, regime shifts, and temporally varying noise that degrade the robustness of standard regression models. We introduce the Variability Aware Recursive (VARNN) Neural Network, a novel residual-aware architecture for supervised time series regression that learns an explicit error memory from recent prediction residuals and uses it to recalibrate subsequent predictions. VARNN augments a feed-forward predictor with a learned error memory state that is updated from residuals over a short context steps as a signal of variability and drift, and then conditions the final prediction at the current time. We study four configurations along two orthogonal axes: (i) residual memory as instantaneous (RM) an embedding of the current innovation only, or accumulative (ARM) that augment current innovation with past residual memory states to capture drift and volatility bursts; and (ii) the presence or absence of an activation memory (AM), which carries the previous latent activation to enrich short-term temporal dynamics and stabilize predictions under noise. Across nine datasets from three important domains, Energy, Healthcare, and Environmental monitoring, experimental results demonstrate that VARNN achieves superior performance and attains the lowest test MSE with minimal computational overhead over static, dynamic, and sequence baselines. Our findings show that the VARNN model offers robust predictions under a drift and volatility environment, highlighting its potential as a promising framework for time series learning.

Primary Area: learning on time series and dynamical systems

Submission Number: 23841