ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations

Chang Liu, Bohao Zhao, Jingtao Ding, Yong Li

Keywords: Chaotic Systems, Foundation Model, Zero-shot Generalization

TL;DR: We introduce a foundation model on a vast and diverse corpus of thousands of chaotic systems, enabling zero-shot forecasting for previously unseen chaotic systems.

Abstract: Accurately forecasting chaotic systems, prevalent in domains including weather prediction and fluid dynamics, remains a significant scientific challenge. The inherent sensitivity of these systems to initial conditions, coupled with a scarcity of observational data, severely constrains traditional modeling approaches. Since these models are typically trained for specific systems, they lack zero-shot or few-shot capabilities on novel or data-limited scenarios. While emerging foundation models address this via pretraining on multiple systems, existing architectures typically operate at a single resolution, often failing to capture the intrinsic multi-scale temporal structures where distinct dynamical patterns unfold. To overcome this limitation, we introduce ChaosNexus, a universal forecasting model driven by our ScaleFormer architecture. It explicitly captures the multi-scale structure of chaotic dynamics with a U-Net-inspired design, enabling the simultaneous modeling of fine-grained fluctuations and coarse-grained trends. Augmented with Mixture-of-Experts layers and a wavelet-based frequency fingerprint, the model can generalizes across heterogeneous dynamical regimes. On a large-scale testbed comprising over 9,000 synthetic chaotic systems, it demonstrates notable improvements in the fidelity of long-term attractor statistics while achieving competitive point-wise forecasting accuracy compared to the leading baseline. This robust performance extends to real-world applications with exceptional data efficiency. For instance, in 5-day global weather forecasting, ChaosNexus achieves a competitive zero-shot mean error below 1°C, a result that further improves with few-shot fine-tuning. Moreover, experiments on the scaling behavior of ChaosNexus provide a guiding principle for scientific foundation models: cross-system generalization stems from the diversity of training systems, rather than sheer data volume.

Primary Area: learning on time series and dynamical systems

Submission Number: 17691