Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Entropy Estimates for Generative Models
Kirill Struminsky, Michael Figurnov, Dmitry Vetrov
Feb 12, 2018 (modified: Feb 20, 2018)ICLR 2018 Workshop Submissionreaders: everyone
Abstract:Different approaches to generative modeling entail different approaches to evaluation. While some models admit test likelihood estimation, for others only proxy metrics for visual quality are being reported. In this paper, we propose a simple method to compute differential entropy of an arbitrary decoder-based generative model. Using this approach, we found that models with qualitatively different samples are distinguishable in terms of entropy. In particular, adversarially trained generative models typically have higher entropy than variational autoencoders. Additionally, we provide support for the application of entropy as a measure of sample diversity.
TL;DR:A simple differential entropy estimator applied to comparison of generative models