Adaptive Sampling for Probabilistic Forecasting under Distribution Shift

Luca Masserano, Syama Sundar Rangapuram, Shubham Kapoor, Rajbir Singh Nirwan, Youngsuk Park, Michael Bohlke-Schneider

Keywords: Data distribution shift, probabilistic forecasting, adaptive sampling, importance sampling, Bayesian optimization

TL;DR: We introduce a two-step, model-agnostic method with Bayesian optimization to forecast under distribution shift.

Abstract:

The world is not static: This causes real-world time series to change over time
through external, and potentially disruptive, events such as macroeconomic cycles
or the COVID-19 pandemic. We present an adaptive sampling strategy that selects
the part of the time series history that is relevant for forecasting. We achieve this by
learning a discrete distribution over relevant time steps by Bayesian optimization.
We instantiate this idea with a two-step method that is pre-trained with uniform
sampling and then training a lightweight adaptive architecture with adaptive sam-
pling. We show with synthetic and real-world experiments that this method adapts
to distribution shift and significantly reduces the forecasting error of the base model
for three out of five datasets.