Representations of Computer Programs in the Human Brain

Shashank Srikant, Benjamin Lipkin, Anna A Ivanova, Evelina Fedorenko, Una-May O'Reilly

Keywords: ML for PL/SE, ML models of code, Code representations, Brain representations, Cognitive neuroscience, Multivoxel pattern analysis, Representation decoding analysis, Representation similarity analysis, fMRI analysis

Abstract: We present the first study relating representations of computer programs generated by unsupervised machine learning (ML) models and representations of computer programs in the human brain. We analyze recordings---brain representations---from functional magnetic resonance imaging (fMRI) studies of people comprehending Python code. We discover brain representations, in different and specific regions of the brain, that encode static and dynamic properties of code such as abstract syntax tree (AST)-related information and runtime information. We also map brain representations to representations of a suite of ML models that vary in their complexity. We find that the Multiple Demand system, a system of brain regions previously shown to respond to code, contains information about multiple specific code properties, as well as machine learned representations of code. We make all the corresponding code, data, and analysis publicly available.

One-sentence Summary: An analysis to determine whether ML models trained on code corpora learn the same information that our brains learn when we comprehend code.