ATM: Adaptive Time Series Tokenization with Semantic Modeling

Sunrui Zhou, Haomin Yu, Mo Saraee, Jilin Hu

Keywords: time series forecasting; patching; adaptive tokenization; semantic modeling

TL;DR: We propose ATM, an adaptive time-series patching framework that incorporates latent semantic information for multivariate time-series forecasting.

Abstract: Recent advances in time series forecasting have achieved remarkable success, yet two critical challenges remain underexplored: the limitations of fixed-length patching strategies and the lack of semantic-level modeling. Fixed-length patches struggle to capture heterogeneous temporal patterns and often truncate temporal patterns, while existing methods largely rely on data-driven statistical patterns without semantic guidance. We propose Adaptive Time Series Tokenization with Semantic Modeling (ATM), a novel framework designed to address these issues. ATM introduces a Temporal Tokenization Module, which consists of two interrelated components: the Time Tokenizer, which adaptively partitions time series to preserve meaningful patterns (e.g., full cycles or peaks), and the Semantic Tokenization Regularization, designed to ensure semantically coherent temporal partitioning. In addition, ATM incorporates Semantic-Aware Modeling, where a Semantic Extractor enriches patches with latent semantic information and a Semantic Modeler captures hierarchical dependencies from local temporal patches to global sequence structures, enhanced by a Mixture-of-Experts module for diverse pattern modeling. Extensive experiments show that ATM consistently surpasses state-of-the-art methods in both long-term and short-term forecasting, demonstrating its effectiveness and strong generalization ability. Code is available at https://anonymous.4open.science/r/ATM.

Primary Area: learning on time series and dynamical systems

Submission Number: 12107