Treeformer: Deep Tree-Based Model with Two-Dimensional Information Enhancement for Multivariate Time Series Forecasting

Xinhe Liu, Wenmin Wang

Abstract: Driven by real-world demands of processing massive high-frequency data and achieving longer forecasting horizons in time series forecasting scenarios, a variety of deep learning architectures designed for time series forecasting have emerged at a rapid pace. However, this rapid development actually leads to a sharp increase in parameter size, and the introduction of numerous redundant modules typically offers only limited contribution to improving prediction performance. Although prediction models have shown a trend towards simplification over a period, significantly improving prediction performance, they remain weak in capturing dynamic relationships. Moreover, the predictive accuracy depends on the quality and extent of data preprocessing, making them unsuitable for handling complex real-world data. To address these challenges, we introduced Treeformer, an innovative model that treats the traditional tree-based machine learning model as an encoder and integrates it with a Transformer-based forecasting model, while also adopting the idea of time–feature two-dimensional information extraction by channel independence and cross-channel modeling strategy. It fully utilizes the rich information across variables to improve the ability of time series forecasting. It improves the accuracy of prediction on the basis of the original deep model while maintaining a low computational cost and exhibits better applicability to real-world datasets. We conducted experiments on multiple publicly available datasets across five domains—electricity, weather, traffic, the forex market, healthcare. The results demonstrate improved accuracy, and provide a better hybrid approach for enhancing predictive performance in Long-term Sequence Forecasting (LSTF) problems.

External IDs:doi:10.3390/math13172818