Phonological learning and encoding with small RNNs

Anonymous

Keywords: RNN phonology, encoder-decoder representations, phone embeddings

TL;DR: Small RNNs learn to do vowel harmony and induce linguistically meaningful structure in embedding spaces.

Abstract: Motivated by recent successes in applying large recurrent neural networks to the task of learning phonological phenomena, we set out to investigate whether a small (orders of magnitude smaller by parameter count) RNN could similarly learn a (morpho)phonologically non-trivial task, and whether the representations learned will be linguistically informative. We demonstrate that such a network can learn vowel harmony, including patterns of transparent neutrality, and moreover that an extremely small (2 orders of magnitude smaller than is typically used) phone embedding space is able to induce linguistically meaningful latent structure.