Evaluating Foundation Models for the Brain: A Dynamical Systems Perspective

Max Kanwal

Keywords: dynamical systems, OOD generalization, scaling

TL;DR: The perspective paper introduces a framework for evaluating brain foundation models through a dynamical-systems lens, incorporating generalization theory, brain-specific distribution shifts, and benchmarking to identify genuinely foundational models.

Abstract: Foundation models promise to transform neuroscience and brain–computer interfaces (BCIs), but their evaluation remains fragmented and often misleading. Standard benchmarks that emphasize in-distribution accuracy fail to capture what truly matters in dynamical domains: the ability to generalize across conditions that differ from those seen during training. In this perspective, we propose a unified framework for evaluating brain foundation models through the lens of dynamical systems theory. We introduce a generalization spectrum—a hierarchy of distribution shifts spanning system identity, parameter regimes, attractor structure, initial conditions, and observation noise—that clarifies what kinds of robustness should be expected from models claiming to be ``foundational.'' We then map this spectrum onto a brain-specific taxonomy of distribution shifts—surface (hardware and noise), functional (state and task), and structural (subject and species)—to ground the framework in neuroscientific practice. Building on this foundation, we outline success criteria for brain foundation models: strong out-of-distribution generalization with minimal data, benchmark-validated performance across diverse tasks, and scalable power-law improvements with model and dataset size. This framework provides a common language for machine learning, neuroscience, and clinical research, and offers a roadmap for building evaluation cultures that distinguish narrow task solvers from truly foundational models.

Submission Number: 79