RPNT: Robust Pre-trained Neural Transformer - A Pathway for Generalized Motor Decoding

ICLR 2026 Conference Submission15722 Authors

Keywords: brain decoding

TL;DR: Our work proposed a new neural transformer architecture that can handle different recording configurations and neural nonstationary activity, which achieved good decoding generalization.

Abstract: Brain decoding aims to interpret and translate neural activity into behaviors. As such, it is imperative that decoding models are able to generalize across variations, such as recordings from different brain sites, distinct sessions, different types of behavior, and a variety of subjects. Current models can only partially address these challenges and warrant the development of pretrained neural transformer models capable to adapt and generalize. In this work, we propose RPNT - Robust Pretrained Neural Transformer, designed to achieve robust generalization through pretraining, which in turn enables effective finetuning given a downstream task. To achieve the proposed architecture of RPNT, we undertook an investigation to determine which building blocks will be suitable for neural spike activity modeling, since components from transformer models developed for other modalities do not transfer directly to neural data. In particular, RPNT unique components include 1) Multidimensional rotary positional embedding (MRoPE) to aggregate experimental metadata such as site coordinates, session name and behavior types; 2) Context-based attention mechanism via convolution kernels operating on global attention to learn local temporal structures for handling non-stationarity of neural population activity; 3) Robust self-supervised learning (SSL) objective with uniform causal masking strategies and contrastive representations. We pretrained two separate versions of RPNT on distinct datasets a) Multi-session, multi-task, and multi-subject microelectrode benchmark; b) Multi-site recordings using high-density Neuropixel 1.0 probes. The datasets include recordings from the dorsal premotor cortex (PMd) and from the primary motor cortex (M1) regions of nonhuman primates (NHPs) as they performed reaching tasks. After pretraining, we evaluated the generalization of RPNT in cross-session, cross-type, cross-subject, and cross-site downstream behavior decoding tasks. Our results show that RPNT consistently achieves and surpasses the decoding performance of existing decoding models in all tasks. Our ablation and sweeping analysis demonstrate the necessity and robustness of the proposed novel components.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 15722