A Continuous Approach to Metaphorically Motivated Regular Polysemy in Language Models

Anna Temerko

Keywords: polysemy, regular polysemy, metaphor, language models, surprisal, semantic similarity, BERT, Llama

TL;DR: An investigation on regular polysemy representation in language models with a focus on graded regularity and metaphorically motivated sense extensions.

Abstract: Linguistic accounts show that a word's polysemy structure is largely governed by systematic sense alternations that form overarching patterns across the vocabulary. While psycholinguistic studies confirm the psychological validity of regularity in human language processing, in the research on large language models (LLMs) this phenomenon remains largely unaddressed. Revealing models’ sensitivity to systematic sense alternations of polysemous words can give us a better understanding of how LLMs process ambiguity and to what extent they emulate representations in the human mind. For this, we employ the measures of surprisal and semantic similarity as proxies of human judgment on the acceptability of novel senses. We focus on two aspects that have not received much attention previously —metaphorically motivated patterns and the continuous nature of regularity. We find evidence that surprisal from language models represents regularity of polysemic extensions in a human-like way, discriminating between different types of senses and varying regularity degrees, and overall strongly correlating with human acceptability scores.

Submission Number: 168