Modeling Emergent Lexicon Formation with a Self-Reinforcing Stochastic ProcessDownload PDF

Published: 25 Mar 2022, Last Modified: 05 May 2023EmeCom Workshop at ICLR 2022Readers: Everyone
Keywords: emergent language, model, entropy, stochastic process
TL;DR: We introduce a self-reinforcing stochastic process as a theoretical model of lexicon entropy in emergent language experiments.
Abstract: We introduce FiLex, a self-reinforcing stochastic process which models finite lexicons in emergent language experiments. The central property of FiLex is that it is a self-reinforcing process, parallel to the intuition that the more a word is used in a language, the more its use will continue. As a theoretical model, FiLex serves as a way to both explain and predict the behavior of the emergent language system. We empirically test FiLex's ability to capture the relationship between the emergent language's hyperparameters and the lexicon's Shannon entropy.
3 Replies