Does multimodal pre-activation influence linguistic expectations in LLMs and humans?

Published: 03 Oct 2025, Last Modified: 03 Oct 2025CPL 2025 SpotlightPosterEveryoneRevisionsBibTeXCC BY 4.0
Keywords: multimodality, language models, sentence processing, surprisal, reading times, word meaning, grounded semantics
TL;DR: When "watch" is highly expected in a context, LLMs do not show lower surprisal for "compass" (unexpected in context, visually similar to a watch) compared to "dog" (unexpected, dissimilar visually). We are testing if humans do.
Submission Number: 10
Loading