Predicate-Argument Relations in the Human Brain

Yipei Wang, Xiaoqian Wang, Ronnie Bring Wilbur, Jeffrey Mark Siskind

Keywords: compositionality, thematic role assignment, neuroimaging, fMRI

TL;DR: Cross modal and cross subject decoding of who did what to whom from fMRI data elicited from video and text stimuli, involving common subsets of voxels, suggests a common neural substrate.

Abstract:

Humans perceive, understand, and describe the world in terms of
relations between things: Who did what to whom. Novel stimulus design,
neuroimaging (fMRI) data, and machine-learning analysis methods allow
assessing the degree to which the human brain represents this
information in a way that can be decoded across different subjects and
modalities. Analysis of the voxels involved in this decoding
demonstrates signifcant commonality across pairs of subjects and
modalities. This suggests a shared neural substrate that supports
predicate-argument relations in multiple modalities that is common
across different people.

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 982