Temporal Eigenstate Networks: Linear-Complexity Sequence Modeling via Spectral Decomposition

Oluwatosin Abioye Afolabi

Keywords: deep learning, transfromers, neural networks

TL;DR: We introduce Temporal Eigenstate Networks (TEN), a novel architecture for sequence mod- eling that achieves O(T ) complexity compared to the O(T 2) complexity of transformer attention mechanisms

Abstract:

\begin{abstract}
We introduce Temporal Eigenstate Networks (TEN), a novel architecture for sequence modeling that achieves $O(T)$ complexity compared to the $O(T^2)$ complexity of transformer attention mechanisms, where $T$ is the sequence length. TEN operates by decomposing temporal dynamics into learned eigenstate superpositions that evolve through complex-valued phase rotations, enabling efficient capture of both short and long-range dependencies. Our approach combines insights from spectral operator theory, state-space models, and Hamiltonian dynamics to create a mathematically principled framework that eliminates the attention bottleneck—surpassing even hardware-optimized attention mechanisms at scale. On benchmark tasks, TEN achieves 3-28× speedup over transformers on sequences of length 512-8192 while using 120× less memory, with competitive or superior accuracy on language modeling, long-range reasoning, and time-series prediction. We provide theoretical analysis proving universal approximation capabilities, stability guarantees, and efficient gradient flow. TEN opens a new direction for scalable sequence modeling, promising for extremely long contexts (1M+ tokens), edge deployment, and AGI systems requiring efficient temporal reasoning.
\end{abstract}

Submission Number: 3