Modeling Emergent Lexicon Formation with a Self-Reinforcing Stochastic Process

Brendon Boldt, David R Mortensen

Keywords: emergent language, model, entropy, stochastic process

TL;DR: We introduce a self-reinforcing stochastic process as a theoretical model of lexicon entropy in emergent language experiments.

Abstract: We introduce FiLex, a self-reinforcing stochastic process which models finite lexicons in emergent language experiments. The central property of FiLex is that it is a self-reinforcing process, parallel to the intuition that the more a word is used in a language, the more its use will continue. As a theoretical model, FiLex serves as a way to both explain and predict the behavior of the emergent language system. We empirically test FiLex's ability to capture the relationship between the emergent language's hyperparameters and the lexicon's Shannon entropy.