A Comparative Empirical Study of Relative Embedding Alignment in Neural Dynamical System Forecasters

Deniz Kucukahmetler, Maximilian Jean Hemmann, Julian Mosig von Aehrenfeld, Maximilian Amthor, Christian Deubel, Nico Scherf, Diaaeldin Taha

Keywords: dynamical systems, representational alignment

Abstract: We study neural forecasters for dynamical systems through the lens of representational alignment. We introduce anchor-based, geometry-agnostic \emph{relative embeddings} that remove rotational and scaling ambiguities, enabling robust cross-seed and cross-architecture comparison. Across diverse periodic, quasi-periodic, and chaotic systems, we observe consistent family-level patterns: MLPs align with MLPs, RNNs with RNNs, and ESNs show reduced alignment on chaotic dynamics, while Transformers often align weakly but still perform well. Alignment generally correlates with forecasting accuracy, yet high accuracy can coexist with low alignment. Relative embeddings thus offer a simple, reproducible basis for comparing learned dynamics.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 48