Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Wan Zhang, Qinjie Lin, Weijian Li, Chan Lee, Kai Zhang, Han Liu

Keywords: Time Series Forecasting, Foundation Models, Sparsity Pattern Aggregation

Abstract: Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information. We introduce Hopformer (Homogeneity-Pursuit Transformer), a two-stage forecasting framework that addresses this challenge. In the first stage, our novel Sparsity Pattern Aggregation (SPA) scheme extracts a common, low-variance trend incorporating the covariates. This acts as a homogenization layer. In the second stage, a LoRA-fine-tuned Transformer models the remaining complex dependencies in the residual signals. Our method is theoretically grounded. We prove that SPA achieves a near-optimal bias-variance trade-off via an oracle inequality. We also provide generalization bounds for the second stage under dependent time series via an information-theoretic analysis. Hopformer sets a new state of the art, improving the MASE by an average of 6.56\% on both synthetic and real-world benchmarks.

Primary Area: learning on time series and dynamical systems

Submission Number: 14992