Rapid Model Comparison by Amortizing Across Models

Lily H. Zhang, Michael C. Hughes

Keywords: Variational inference, amortized inference, topic models, model comparison

TL;DR: We develop VAEs where the encoder takes a model parameter vector as input, so we can do rapid inference for many models

Abstract: Comparing the inferences of diverse candidate models is an essential part of model checking and escaping local optima. To enable efficient comparison, we introduce an amortized variational inference framework that can perform fast and reliable posterior estimation across models of the same architecture. Our Any Parameter Encoder (APE) extends the encoder neural network common in amortized inference to take both a data feature vector and a model parameter vector as input. APE thus reduces posterior inference across unseen data and models to a single forward pass. In experiments comparing candidate topic models for synthetic data and product reviews, our Any Parameter Encoder yields comparable posteriors to more expensive methods in far less time, especially when the encoder architecture is designed in model-aware fashion.