Lightweight Online Adaption for Time Series Foundation Model Forecasts

Thomas L Lee, William Toner, Rajkarn Singh, Artjom Joosen, Martin Asenov

TL;DR: We propose ELF a model-agnostic and efficient method to improve the forecasts of time series foundation models at deployment by using online feedback.

Abstract: Foundation models (FMs) have emerged as a promising approach for time series forecasting. While effective, FMs typically remain fixed during deployment due to the high computational costs of learning them online. Consequently, deployed FMs fail to adapt their forecasts to current data characteristics, despite the availability of online feedback from newly arriving data. This raises the question of whether FM performance can be enhanced by the *efficient* usage of this feedback. We propose *ELF* to answer this question. ELF is a lightweight mechanism for the online adaption of FM forecasts in response to online feedback. ELF consists of two parts: **a)** the *ELF-Forecaster* which is used to learn the current data distribution; and **b)** the *ELF-Weighter* which is used to combine the forecasts of the FM and the ELF-Forecaster. We evaluate the performance of ELF in conjunction with several recent FMs across a suite of standard time series datasets. In *all* of our experiments we find that using ELF improves performance. This work demonstrates how efficient usage of online feedback can be used to improve FM forecasts.

Lay Summary:

One common form of data in the real world are values that change over time. For example, the temperature in a city. Data of this form are called *time series*. One problem of interest is *forecasting* the future of a time series. For instance, predicting future temperatures. In this work we look at a particular type of model to forecast time series: *foundation models* (FMs), large neural networks similar in form to large language models like ChatGPT. While FMs have been shown to forecast time series well, by looking at the new values of time series as we progress through time we should be able to improve a model’s forecasts.

We propose a method *ELF* to improve FM forecasts as we see more of the time series. ELF works by learning another small forecaster on the new data of the time series and then adapts the FM forecast by combining it with the forecast of this other forecaster. An important property of the method is that it is fast enough that it can be used between the times where you need to produce a forecast. We found in our experiments that it works well, improving performance of FM forecasts. By improving the performance of forecasts, ELF takes a small step towards improving our ability predict real time series, from future temperature in a city to future demand for a utility, enabling better decision making.

Primary Area: Deep Learning->Sequential Models, Time series

Keywords: Time Series, Foundation Models, Deep Learning, Continual Learning, Online Learning

Submission Number: 2680