SPADE-S: A Sparsity-Robust Foundational Forecaster

Malcolm Wolff, Matthew Li, Ravi Kiran Selvam, Hanjing Zhu, Kin G. Olivares, Ruijun Ma, Abhinav Katoch, Shankar Ramasubramanian, Mengfei Cao, Roberto Bandarra, Rahul Gopalsamy, Stefania La Vattiata, Sitan Yang, Michael W. Mahoney

Keywords: forecasting

Abstract: Despite significant advancements in time series forecasting, accurate modeling of time series with strong heterogeneity in magnitude and/or sparsity patterns remains challenging for state of the art deep learning architectures. We identify several factors that lead existing models to systematically under-perform on low magnitude and sparse time series, including loss functions with implicit biases toward high-magnitude series, training-time sampling methods, and limitations of time series encoding methods. To address these limitations, we introduce SPADE-S, a robust forecasting architecture with a novel multi-head convolutional encoder and a model arm specifically designed to handle sparse multi-variate time series. SPADE-S significantly reduces magnitude and sparsity-based systematic biases and improves overall prediction accuracy; empirical results demonstrate that SPADE-S outperforms existing state-of-the-art approaches across a diverse set of use-cases in demand forecasting. In particular, we show that, depending on the quantile forecast and magnitude of the series, SPADE-S can improve forecast accuracy by up to 15%. This results in P90 overall forecast accuracy gains of 2.21%, 6.58%, and 4.28%, and P50 forecast accuracy gains of 0.92%, 0.77%, and 1.95% respectfully, for each of three distinct datasets, ranging from 3 million to 700 million series, from a large online retailer.

Submission Number: 30