NeuMa, Born to Work

Yuanda Yang, HAORAN ZHANG, Yuehui Li

Keywords: State Space Models, Computational Neuroscience, Hippocampal Modeling, Interpretability, Architectural Priors

TL;DR: We reframe Mamba as an incomplete model of the hippocampus and demonstrate that a more neuro-faithful architecture unlocks superior algorithmic reasoning, emergent robustness, and profound biological fidelity.

Abstract: Grounded in the evolutionary principle that resource constraints favor structural solutions for complex computation, we propose a neuro-centric framework that reframes the success of State Space Models (SSMs), e.g. Mamba, as an unconscious convergence to an incomplete model of the hippocampus—the brain's canonical circuit for sophisticated computations such as pattern separation and completion. Motivated by this, we introduce NeuroMamba (NeuMa), a novel architecture that consciously and faithfully implements the canonical hippocampal circuit, including the dentate gyrus (DG), Cornu Ammonis 3 (CA3), and Cornu Ammonis 1 (CA1), using foundational SSM blocks. Enabled by custom kernels, our design bridges the gap between biological plausibility and practical efficiency. Experiments demonstrate that NeuMa achieves superior performance and learning efficiency on synthetic benchmarks. More critically, it exhibits profound biological fidelity by spontaneously replicating the "orthogonalized state machine" dynamics of the biological hippocampus. Finally, we validate its capacity for real-world scientific discovery by developing a generative agent for piezoelectric catalysis that achieves superior performance in this complex, low-resource domain, thereby showcasing a new path for AI architecture design rooted in neuroscience.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 10846