Diffusion-LLM Provides Ultra-Long-Term Time Series Forecasting with Probabilistic Alignment

ICLR 2026 Conference Submission18950 Authors

Keywords: Multimodal Learning, Time-Series Data/streams, Large Multimodal Models, ML Applications

Abstract: Time series forecasting is a fundamental task in machine learning. Recently, Large Language Models (LLMs) have gained attention for this task due to their strong generalization capabilities, particularly in recognizing patterns and performing complex reasoning across diverse data modalities. Apart from having the architecture suitable for long-context learning, LLMs are an interesting option also because of their few-shot and zero-shot transfer learning capability, making it possible to use pretrained frozen LLMs directly for time series forecasting. However, challenges remain in adapting LLMs to multimodal tasks: they often lack a calibrated understanding of probabilistic structure in non-text modalities and struggle with aligning heterogeneous representations. To address these limitations, we propose Diffusion-LLM, a novel framework that integrates a conditional diffusion model into an LLM-based forecasting pipeline. This joint setup enables the model to learn the conditional distribution of future time series trajectories while reinforcing semantic alignment in the shared latent space. We evaluate Diffusion-LLM on six standard long-term forecasting benchmarks, including ETT, Weather, and ECL datasets. Our approach consistently outperforms existing LLM-based baselines, achieving substantial gains in ultra-long-term and few-shot forecasting tasks, while demonstrating the effectiveness of distribution-aware regularization for enhancing the robustness and generalization of time series LLMs.

Primary Area: learning on time series and dynamical systems

Submission Number: 18950