WeightUpdater¶

- class gmmvi.optimization.gmmvi_modules.weight_updater.WeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)[source]¶
This class provides a common interface for updating the weights of the mixture model.
It currently supports two options:
The
DirectWeightUpdater
straightforwardly applies a natural gradient update using the given stepsize.The
TrustRegionBasedWeightUpdater
treats the stepsize as a trust-region constraint between the current distribution over weights and the updated distribution, and performs the largest step in the direction of the natural gradient that confines to this constraint.
- Parameters:
gmm_wrapper –
GmmWrapper
The wrapped model where we want to update the weights.temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.
use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.
- static build_from_config(config, gmm_wrapper)[source]¶
This static method provides a convenient way to create a
DirectWeightUpdater
orTrustRegionBasedWeightUpdater
depending on the provided config.- Parameters:
config – dict The dictionary is typically read from YAML a file, and holds all hyperparameters.
gmm_wrapper –
GmmWrapper
The wrapped model for which we want to update the weights.
- update_weights(samples: Tensor, background_mixture_densities: Tensor, target_lnpdfs: Tensor, stepsize: float)[source]¶
Computes the importance weights and uses them to estimate the natural gradient. Performs a natural gradient step using the given stepsize.
- Parameters:
samples – tf.Tensor The samples for which the background_mixture_densities and target_lnpdfs were evaluated. Needed for computing the importance weights.
background_mixture_densities – tf.Tensor The log_densities of the samples for the distribution that was effectively used for obtain the provided samples. Needed for computing the importance weights.
target_lnpdfs – tf.Tensor The log densities of the target distribution evaluated for the provided samples,
.
stepsize – float The stepsize that should be used for performing the weight update.
DirectWeightUpdater¶
- class gmmvi.optimization.gmmvi_modules.weight_updater.DirectWeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)[source]¶
This class can be used for directly updating the weights along the natural gradient, using the given stepsize.
- Parameters:
gmm_wrapper –
GmmWrapper
The wrapped model where we want to update the weights.temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.
use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.
- _update_weights_from_expected_log_ratios(expected_log_ratios: Tensor, stepsize: tf.float32)[source]¶
Directly uses the stepsize to update the weights towards the expected_log_ratios
- Parameters:
expected_log_ratios – tf.Variable(tf.float32) A vector containing an (MC-)estimate of
, for every component o.
stepsize – tf.float32 The stepsize
, the new weights are proportional to
.
TrustRegionBasedWeightUpdater¶
- class gmmvi.optimization.gmmvi_modules.weight_updater.TrustRegionBasedWeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)[source]¶
This class can be used for performing the weight update by treating the stepsize as a KL constraint.
Constrains the KL between the updated weights and the current weights
.
- Parameters:
gmm_wrapper –
GmmWrapper
The wrapped model where we want to update the weights.temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.
use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.
- _bracketing_search(expected_log_ratios: Tensor, kl_bound: tf.float32, lower_bound: tf.float32, upper_bound: tf.float32) [tf.float32, tf.float32, <class 'tensorflow.python.framework.ops.Tensor'>] [source]¶
This method finds the largest stepsize eta, such that the updated weight distribution stays within a KL-constrained trust-region around the current distribution. Here we use a simple bracketing search, which can be efficiently performed within a Tensorflow graph. The procedure simple keeps track of a lower bound and an upper bound on the optimal stepsize and recursively evaluates the arithmetic mean of both bounds. If this mean-stepsize results in a too large KL divergence, it becomes the new lower bound; otherwise the new upper bound.
- Parameters:
expected_log_ratios – tf.Tensor A vector containing an (MC-)estimate of
, for every component o.
kl_bound – tf.float32 The trust region constraint
lower_bound – tf.float32 The initial lower bound on the stepsize
upper_bound – The initial upper bound on the stepsize
- Returns:
new_lower_bound - The lower bound after a stopping criterion was reached.
new_upper_bound - The upper bound after a stopping criterion was reached.
new_log_weights - log of the updated weights,
.
- Return type:
tuple(tf.float32, tf.float32, tf.Tensor)
- _update_weights_from_expected_log_ratios(expected_log_ratios, kl_bound)[source]¶
Perform the weight update, treating the stepsize as constraint on the KL-divergence.
- Parameters:
expected_log_ratios – tf.Variable(tf.float32) A vector containing an (MC-)estimate of
, for every component o.
stepsize – tf.float32 The stepsize
, the new weights will satisfy
.
- kl(eta: tf.float32, component_rewards: Tensor) [tf.float32, <class 'tensorflow.python.framework.ops.Tensor'>] [source]¶
Computes the Kullback-Leibler divergence between the updated component and current component, when updating with stepsize eta along the natural gradient.
- Parameters:
eta – tf.float32 The stepsize for which the KL divergence should be computed.
component_rewards – tf.float32 A tensor containing an MC-estimate of the expected reward (expected logratios) of each component,
- Returns:
kl - a float corresponding to the KL between the updated and previous weight distribution.
new_log_weights - log of the updated weights,
.
- Return type: