Temporal Query Network for Efficient Multivariate Time Series Forecasting

Shengsheng Lin, Haojun Chen, Haijie Wu, Chunyun Qiu, Weiwei Lin

TL;DR: This paper introduces Temporal Query Network (TQNet), a lightweight model with a novel Temporal Query technique that efficiently captures multivariate correlations.

Abstract: Sufficiently modeling the correlations among variables (aka channels) is crucial for achieving accurate multivariate time series forecasting (MTSF). In this paper, we propose a novel technique called Temporal Query (TQ) to more effectively capture multivariate correlations, thereby improving model performance in MTSF tasks. Technically, the TQ technique employs periodically shifted learnable vectors as queries in the attention mechanism to capture global inter-variable patterns, while the keys and values are derived from the raw input data to encode local, sample-level correlations. Building upon the TQ technique, we develop a simple yet efficient model named Temporal Query Network (TQNet), which employs only a single-layer attention mechanism and a lightweight multi-layer perceptron (MLP). Extensive experiments demonstrate that TQNet learns more robust multivariate correlations, achieving state-of-the-art forecasting accuracy across 12 challenging real-world datasets. Furthermore, TQNet achieves high efficiency comparable to linear-based methods even on high-dimensional datasets, balancing performance and computational cost. The code is available at: https://github.com/ACAT-SCUT/TQNet.

Lay Summary: Predicting what will happen in the future based on patterns from the past (known as time series forecasting) is important in many areas like weather prediction, traffic control, and energy management. These tasks often involve many related measurements (such as temperatures at different locations or traffic at different intersections), and understanding how these measurements influence each other is key to making accurate predictions. In this paper, we introduce a new method called Temporal Query (TQ) that helps computers better understand the relationships between different measurements over time. TQ works by learning to focus on long-term patterns across variables, even when individual data points might be noisy or incomplete. Based on this method, we build a lightweight model named TQNet, which is simple but surprisingly powerful. TQNet outperforms many existing forecasting models on 12 real-world datasets while staying fast and efficient, even when working with large and complex data. This makes it a practical and effective tool for a wide range of forecasting applications.

Link To Code: https://github.com/ACAT-SCUT/TQNet

Primary Area: Applications->Time Series

Keywords: Multivariate time series forecasting, correlation modeling, time series analysis, machine learning

Submission Number: 1177