On the Dynamics of Coherent Memory Structures in Neural Fields

ICLR 2026 Conference Submission21675 Authors

Keywords: deep learning, traveling waves, neural oscillators, mori-zwanzig formalism, projection operator formalism

TL;DR: We show that coupled dynamics between memory encoding and decoding give rise to emergent dynamical modes.

Abstract: Memory in biological neural networks emerges as coherent structures--spatiotemporal waves and manifold trajectories--driven by complex synaptic activities across neural fields. By contrast, many artificial neural networks--from gated recurrent units to recent state-space-models--remain black-box mechanisms. Recent works provide interpretable latent states by imposing traveling waves or invariant manifolds, but lack data-driven explanatory mechanisms for why such structures should arise. We offer a theoretical framework for studying trivial and emergent coherent dynamics. Building on the Mori-Zwanzig formalism, our approach casts memory as a family of time-dependent projections that reveal how coupled dynamics give rise to memory encoding and decoding. Using this framework, we present a Neural Wave Field architecture that autonomously discovers the memory operator’s leading eigenmodes and leverages them to enhance its long-range memory. We validate our method on both long-range copy benchmarks and chaotic-system forecasting tasks, demonstrating robust long-range accuracy and the spontaneous emergence of interpretable memory modes.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 21675