Word Embedding Uncertainty Estimation

Väinö Yrjänäinen, Isac Boström, Måns Magnusson

Keywords: Word Embeddings, Probabilistic Language Models, Uncertainty Estimation, Approximate Bayesian inference

TL;DR: We introduce Laplace approximation for word embeddings in and show it may perform better than variational inference

Abstract: Probabilistic word embeddings can be used to make scientific inferences from text data. So far, mean-field variational inference is the only Bayesian approach for their uncertainty estimation. Our early results indicate that Laplace Approximation converges faster and provides better uncertainty estimates than mean-field variational inference.

Submission Number: 15