xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories

Maurice Kraus, Felix Divo, Devendra Singh Dhami, Kristian Kersting

Keywords: time series, xLSTM, forecasting, mixing, recurrent

TL;DR: xLSTM-Mixer integrates temporal, joint, and multi-view mixing with xLSTM blocks, achieving consistent SOTA performance in a wide range of long-term forecasting tasks.

Abstract:

Time series data is prevalent across numerous fields, necessitating the development of robust and accurate forecasting models.
Capturing patterns both within and between temporal and multivariate components is crucial for reliable predictions.
We introduce xLSTM-Mixer, a model designed to effectively integrate temporal sequences, joint time-variate information, and multiple perspectives for robust forecasting.
Our approach begins with a linear forecast shared across variates, which is then refined by xLSTM blocks.
They serve as key elements for modeling the complex dynamics of challenging time series data.
xLSTM-Mixer ultimately reconciles two distinct views to produce the final forecast.
Our extensive evaluations demonstrate its superior long-term forecasting performance compared to recent state-of-the-art methods while requiring very little memory.
A thorough model analysis provides further insights into its key components and confirms its robustness and effectiveness.
This work contributes to the resurgence of recurrent models in forecasting by combining them, for the first time, with mixing architectures.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Flagged For Ethics Review: true

Submission Number: 8898