WeightUpdater

Inheritance diagram of gmmvi.optimization.gmmvi_modules.weight_updater.WeightUpdater, gmmvi.optimization.gmmvi_modules.weight_updater.TrustRegionBasedWeightUpdater, gmmvi.optimization.gmmvi_modules.weight_updater.DirectWeightUpdater
class gmmvi.optimization.gmmvi_modules.weight_updater.WeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)[source]

This class provides a common interface for updating the weights of the mixture model.

It currently supports two options:

  1. The DirectWeightUpdater straightforwardly applies a natural gradient update using the given stepsize.

  2. The TrustRegionBasedWeightUpdater treats the stepsize as a trust-region constraint between the current distribution over weights and the updated distribution, and performs the largest step in the direction of the natural gradient that confines to this constraint.

Parameters:
  • gmm_wrapperGmmWrapper The wrapped model where we want to update the weights.

  • temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.

  • use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.

static build_from_config(config, gmm_wrapper)[source]

This static method provides a convenient way to create a DirectWeightUpdater or TrustRegionBasedWeightUpdater depending on the provided config.

Parameters:
  • config – dict The dictionary is typically read from YAML a file, and holds all hyperparameters.

  • gmm_wrapperGmmWrapper The wrapped model for which we want to update the weights.

update_weights(samples: Tensor, background_mixture_densities: Tensor, target_lnpdfs: Tensor, stepsize: float)[source]

Computes the importance weights and uses them to estimate the natural gradient. Performs a natural gradient step using the given stepsize.

Parameters:
  • samples – tf.Tensor The samples for which the background_mixture_densities and target_lnpdfs were evaluated. Needed for computing the importance weights.

  • background_mixture_densities – tf.Tensor The log_densities of the samples for the distribution that was effectively used for obtain the provided samples. Needed for computing the importance weights.

  • target_lnpdfs – tf.Tensor The log densities of the target distribution evaluated for the provided samples, \log p(\mathbf{x}).

  • stepsize – float The stepsize that should be used for performing the weight update.

DirectWeightUpdater

class gmmvi.optimization.gmmvi_modules.weight_updater.DirectWeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)[source]

This class can be used for directly updating the weights along the natural gradient, using the given stepsize.

Parameters:
  • gmm_wrapperGmmWrapper The wrapped model where we want to update the weights.

  • temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.

  • use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.

_update_weights_from_expected_log_ratios(expected_log_ratios: Tensor, stepsize: tf.float32)[source]

Directly uses the stepsize to update the weights towards the expected_log_ratios

Parameters:
  • expected_log_ratios – tf.Variable(tf.float32) A vector containing an (MC-)estimate of \mathbb{E}_{q(\mathbf{x}|o)}\left[ \log \frac{p(\mathbf{x})}{q(\mathbf{x})}\right] , for every component o.

  • stepsize – tf.float32 The stepsize \beta, the new weights are proportional to \exp(\text{old\_log\_weights} + \beta * \text{expected\_log\_ratios}).

TrustRegionBasedWeightUpdater

class gmmvi.optimization.gmmvi_modules.weight_updater.TrustRegionBasedWeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)[source]

This class can be used for performing the weight update by treating the stepsize as a KL constraint.

Constrains the KL between the updated weights and the current weights \text{KL}(q_\text{new}(o) || q(o)).

Parameters:
  • gmm_wrapperGmmWrapper The wrapped model where we want to update the weights.

  • temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.

  • use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.

This method finds the largest stepsize eta, such that the updated weight distribution stays within a KL-constrained trust-region around the current distribution. Here we use a simple bracketing search, which can be efficiently performed within a Tensorflow graph. The procedure simple keeps track of a lower bound and an upper bound on the optimal stepsize and recursively evaluates the arithmetic mean of both bounds. If this mean-stepsize results in a too large KL divergence, it becomes the new lower bound; otherwise the new upper bound.

Parameters:
  • expected_log_ratios – tf.Tensor A vector containing an (MC-)estimate of \mathbb{E}_{q(\mathbf{x}|o)}\left[ \log \frac{p(\mathbf{x})}{q(\mathbf{x})}\right] , for every component o.

  • kl_bound – tf.float32 The trust region constraint

  • lower_bound – tf.float32 The initial lower bound on the stepsize

  • upper_bound – The initial upper bound on the stepsize

Returns:

new_lower_bound - The lower bound after a stopping criterion was reached.

new_upper_bound - The upper bound after a stopping criterion was reached.

new_log_weights - log of the updated weights, \log(q_\text{new}(o)).

Return type:

tuple(tf.float32, tf.float32, tf.Tensor)

_update_weights_from_expected_log_ratios(expected_log_ratios, kl_bound)[source]

Perform the weight update, treating the stepsize as constraint on the KL-divergence.

Parameters:
  • expected_log_ratios – tf.Variable(tf.float32) A vector containing an (MC-)estimate of \mathbb{E}_{q(\mathbf{x}|o)}\left[ \log \frac{p(\mathbf{x})}{q(\mathbf{x})}\right] , for every component o.

  • stepsize – tf.float32 The stepsize \epsilon, the new weights will satisfy \text{KL}(q_\text{new}(o) || q(o)) < \epsilon.

kl(eta: tf.float32, component_rewards: Tensor) [tf.float32, <class 'tensorflow.python.framework.ops.Tensor'>][source]

Computes the Kullback-Leibler divergence between the updated component and current component, when updating with stepsize eta along the natural gradient.

Parameters:
  • eta – tf.float32 The stepsize for which the KL divergence should be computed.

  • component_rewards – tf.float32 A tensor containing an MC-estimate of the expected reward (expected logratios) of each component, R(o)=\mathbb{E}_{q(\mathbf{x}|o)}\left[ \log \frac{p(\mathbf{x})}{q(\mathbf{x})}\right]

Returns:

kl - a float corresponding to the KL between the updated and previous weight distribution.

new_log_weights - log of the updated weights, \log(q_\text{new}(o)).

Return type:

tuple(float, tf.Tensor)