Modeling the Memory-Surpisal Trade-Off over Time: Communicative Efficiency Decreases with Lexico-Grammatical Change in Scientific English

ACL ARR 2025 May Submission3889 Authors

Abstract: The memory-surprisal trade-off (MST) has been shown to hold cross-linguistically as a general principle of communicative efficiency that provides a processing explanation to some basic properties of language. In this paper, we explore the influence of diachronic variation on the MST. We investigate scientific English in the Royal Society Corpus (RSC) spanning from the 18th centure to modern time; to assess the impact of intra-linguistic variation (register), we compare scientific English with ``general language'' using parts of the Corpus of Historical American English (COHA). We observe a clear diachronic effect for scientific English towards decreased efficiency as scientific texts shift from verbal to nominal style and the lexicon in the scientific domain expands, while in general language the effect is less pronounced.

Paper Type: Long

Research Area: Computational Social Science and Cultural Analytics

Research Area Keywords: Computational Social Science and Cultural Analytics, Language Modeling, Linguistic Theories, Cognitive Modeling, and Psycholinguistics, Resources and Evaluation

Contribution Types: Approaches to low-resource settings, Data analysis

Languages Studied: English

Submission Number: 3889