Lag-Llama: Towards Foundation Models for Time Series Forecasting

Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Biloš, Hena Ghonia, Nadhir Hassen, Anderson Schneider, Sahil Garg, Alexandre Drouin, Nicolas Chapados, Yuriy Nevmyvaka, Irina Rish

Keywords: time series, foundational models, probabilistic forecasting, time series forecasting, transformers, decision making, pretrained models

TL;DR: Strong general-purpose univariate probabilistic time-series forecasting model, with power-law analysis to predict the model's scaling behavior

Abstract:

Aiming to build foundation models for time-series forecasting and study their scaling behavior, we present here our work-in-progress on Lag-Llama, a general-purpose univariate probabilistic time-series forecasting model trained on a large collection of time-series data. The model shows good zero-shot prediction capabilities on unseen "out-of-distribution" time-series datasets, outperforming supervised baselines. We use smoothly broken power-laws to fit and predict model scaling behavior. The open source code is made available at
https://github.com/kashif/pytorch-transformer-ts.

Submission Number: 73