Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Optimal Smoothing for Pathwise Adjoints
Jonathan Hüser, Shih-Te Yang, Uwe Naumann
Oct 27, 2017 (modified: Oct 27, 2017)NIPS 2017 Workshop Autodiff Submissionreaders: everyone
Abstract:We propose an optimal smoothing parametrization for gradient estimators of expectations of discontinuous functions. The reparametrization trick with discontinuous functions gives gradient estimators for discrete random variables and makes smoothing applicable in the machine learning context (e.g. variational inference and stochastic neural networks). Our approach is based on an objective that can be solved simultaneously with a primal optimization task. Optimal smoothing is general purpose in the sense that it only requires an extension of the algorithmic differentiation tool without the need to rearrange the model.
TL;DR:We propose an optimal smoothing parametrization for gradient estimators of expectations of discontinuous functions.
Keywords:algorithmic differentiation, automatic differentiation, stochastic backpropagation, monte carlo derivatives, pathwise derivatives
Enter your feedback below and we'll get back to you as soon as possible.