Analyzing the Understanding of Morphologically Complex Words in Large Language Models

Marion Di Marco, Alexander Fraser

Abstract:

We empirically study the ability of a Large Language Model (gpt-3.5-turbo-instruct) to understand morphologically
complex words. In our experiments, we looked at a variety of tasks to analyse German compounds with regard to
compositional word formation and derivation, such as identifying the head noun of existing and novel compounds,
identifying the shared verb stem between two words, or recognizing words constructed with inappropriately used
derivation morphemes as invalid. Our results show that the language model is generally capable of solving most
tasks, except for the task of identifying ill-formed word forms. While the model demonstrated a good overall
understanding of complex words and their word-internal structure, the results also suggest that there is no formal
knowledge of derivational rules, but rather an interpretation of the observed word parts to derive the meaning of a word.