Heterogeneous Graph Temporal Fusion Transformer for Time Series Forecasting in Multi-Domain Physical Systems

ICLR 2026 Conference Submission16485 Authors

Keywords: Heterogeneous Graph, Time Series Forecasting, Multiphysics, Physical Systems, Pre-training, Transformer

TL;DR: HGTFT, a pre-train and fine-tune framework that integrates heterogeneous spatiotemporal data with physics-informed constraints for accurate, physically consistent time series forecasting in Multi-Domain Physical Systems.

Abstract:

Existing Transformer-based models effectively capture multivariate dependencies, while pre-trained large models achieve strong generalization but are often confined to single-object or single-physics settings. Spatial-temporal approaches leverage graph structures but fall short in modeling heterogeneous entities with diverse inter-variable interactions, and they often lack mechanisms to enforce physical consistency.
To address these challenges, we propose the Heterogeneous Graph Temporal Fusion Transformer (HGTFT), a pre-training and fine-tuning framework tailored for spatially and temporally structured physical environments. HGTFT tokenizes observation points and generates embeddings that capture both temporal patterns and spatial correlations, enabling the integration of heterogeneous static and dynamic information.
We further introduce optimized normalization and physics-informed loss functions that enhance predictive accuracy while improving physical plausibility. Applied to temperature, flow, and energy-related datasets in building environments, our approach demonstrates strong zero-shot generalization and achieves substantial accuracy gains through few-shot fine-tuning with domain-specific data.

Primary Area: learning on time series and dynamical systems

Submission Number: 16485