TimeSqueeze: Dynamic Patching for Efficient Long-Context Time Series Forecasting

Sravan Kumar Ankireddy, Nikita Seleznev, Nam H Nguyen, Yulun Wu, Senthil Kumar, Furong Huang, C. Bayan Bruss

Keywords: foundation models, time series forecasting, dynamic patching, conext compression, long-context forecasting

TL;DR: TimeSqueeze combines fine-grained point embeddings with efficient patch embeddings to deliver optimal time series forecasting performance per computational budget on long sequences.

Abstract: Recent progress in time series forecasting has produced large foundation models with strong generalization across domains. However, their effectiveness is constrained by the computational cost of long-context processing. Existing designs face a core trade-off: point-wise embeddings preserve high-frequency information but scale quadratically with sequence length, while patch-based embeddings improve efficiency by downsampling, at the expense of discarding critical temporal details. We present TimeSqueeze, a hybrid forecasting architecture that resolves this trade-off through dynamic input compression. TimeSqueeze employs a two-stage process: (1) a lightweight Mamba encoder extracts fine-grained features at full resolution, and (2) an adaptive patching module assigns smaller patches to information-rich regions and larger patches to redundant segments. This produces a variable-resolution representation that allocates computation where it is most beneficial for forecasting. The compressed sequence is then passed to a Transformer backbone, yielding substantial reductions in token length while retaining critical temporal features. Extensive experiments demonstrate that TimeSqueeze achieves comparable performance with substantially reduced computational cost, or alternatively, enables processing much longer contexts within the same budget for significantly improved accuracy. This results in an average zero-shot MSE reduction of up to 24% compared to equivalent point embedding models, while maintaining similar computational requirements. These results position TimeSqueeze as a scalable and effective architecture for next-generation time series foundation models.

Submission Number: 20