Linguistic Properties and Model Scale in Brain Encoding: From Small to Compressed Language Models

SUBBA REDDY OOTA, Vijay Rowtula, Satya Sai Srinath Namburi GNVV, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi Raju Surampudi

Abstract: Recent work has shown that scaling large language models (LLMs) improves their alignment with human brain activity, yet it remains unclear what drives these gains or which representational properties are responsible. Although larger models often yield better task performance and brain alignment, they are increasingly difficult to analyze mechanistically. This raises a fundamental question: \emph{what is the minimal model capacity required to capture brain-relevant representations?} To address this question, we systematically investigate how constraining model scale and numerical precision affects brain alignment. We compare full-precision LLMs, small language models (SLMs), and compressed variants (quantized and pruned) by predicting fMRI responses during naturalistic language comprehension. Across model families up to 14B parameters, we find that 3B SLMs achieve brain predictivity indistinguishable from larger LLMs, whereas 1B models degrade substantially, particularly in semantic language regions. Brain alignment is remarkably robust to compression: most quantization and pruning methods preserve neural predictivity, with GPTQ as a consistent exception. Linguistic probing reveals a dissociation between task performance and brain predictivity: compression degrades discourse, syntax, and morphology, yet brain predictivity remains largely unchanged. Overall, brain alignment saturates at modest model scales and is resilient to compression, challenging common assumptions about neural scaling and motivating compact models for brain-aligned language modeling.

Lay Summary: When we listen to a story or read a passage, distinctive patterns of activity appear in our brains. Researchers have recently found that modern AI language models produce internal representations that partly resemble these brain patterns in language regions, and that bigger models tend to resemble the brain more closely. This has led many people to assume that ever-larger AI models are needed to understand how the brain processes language. But large models are expensive to run, hard to interpret mechanistically. We asked a simpler question: if the human language system is compact and efficient, what model capacity is actually necessary to predict its neural responses? We compared language models ranging from small (1B parameters) to large (14B), and also tested compressed versions that have been reduced in size to use less memory and computation. We measured how well each model's internal representations matched brain activity recorded while people listened to and read natural stories. We found that surprisingly small models, ~3B parameters, match the brain language processing just as well as much larger ones. Compressing these models only minimally affects their alignment with the brain, even when it noticeably reduces their performance on linguistic tasks. This suggests that the brain-relevant representations of language can be captured by compact, efficient models, opening the door to studying language and the brain with tools that are cheaper, more transparent, and easier to share.

Primary Area: Applications->Neuroscience, Cognitive Science

Keywords: brain encoding, fMRI, light weight language models, larger language models, quantization, linguistic properties, flash-holmes benchmark

Submission Number: 16844