# WeightUpdater

class gmmvi.optimization.gmmvi_modules.weight_updater.WeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)

  This class provides a common interface for updating the weights of the mixture model.

  It currently supports two options:

  1. The `DirectWeightUpdater` straightforwardly applies a natural gradient update using the given stepsize.

  2. The `TrustRegionBasedWeightUpdater` treats the stepsize as a trust-region constraint between the current distribution over weights and the updated distribution, and performs the largest step in the direction of the natural gradient that confines to this constraint.

  Parameters:

    - gmm_wrapper – `GmmWrapper` The wrapped model where we want to update the weights.

    - temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.

    - use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.

  static build_from_config(config, gmm_wrapper)

    This static method provides a convenient way to create a `DirectWeightUpdater` or `TrustRegionBasedWeightUpdater` depending on the provided config.

    Parameters:

      - config – dict The dictionary is typically read from YAML a file, and holds all hyperparameters.

      - gmm_wrapper – `GmmWrapper` The wrapped model for which we want to update the weights.

  update_weights(samples: Tensor, background_mixture_densities: Tensor, target_lnpdfs: Tensor, stepsize: float)

    Computes the importance weights and uses them to estimate the natural gradient. Performs a natural gradient step using the given stepsize.

    Parameters:

      - samples – tf.Tensor The samples for which the background_mixture_densities and target_lnpdfs were evaluated. Needed for computing the importance weights.

      - background_mixture_densities – tf.Tensor The log_densities of the samples for the distribution that was effectively used for obtain the provided samples. Needed for computing the importance weights.

      - target_lnpdfs – tf.Tensor The log densities of the target distribution evaluated for the provided samples, .

      - stepsize – float The stepsize that should be used for performing the weight update.

## DirectWeightUpdater

class gmmvi.optimization.gmmvi_modules.weight_updater.DirectWeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)

  This class can be used for directly updating the weights along the natural gradient, using the given stepsize.

  Parameters:

    - gmm_wrapper – `GmmWrapper` The wrapped model where we want to update the weights.

    - temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.

    - use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.

  _update_weights_from_expected_log_ratios(expected_log_ratios: Tensor, stepsize: tf.float32)

    Directly uses the stepsize to update the weights towards the expected_log_ratios

    Parameters:

      - expected_log_ratios – tf.Variable(tf.float32) A vector containing an (MC-)estimate of , for every component o.

      - stepsize – tf.float32 The stepsize , the new weights are proportional to .

## TrustRegionBasedWeightUpdater

class gmmvi.optimization.gmmvi_modules.weight_updater.TrustRegionBasedWeightUpdater(model: GmmWrapper, temperature: float, use_self_normalized_importance_weights: bool)

  This class can be used for performing the weight update by treating the stepsize as a KL constraint.

  Constrains the KL between the updated weights and the current weights .

  Parameters:

    - gmm_wrapper – `GmmWrapper` The wrapped model where we want to update the weights.

    - temperature – float Usually temperature=1., can be used to scale the importance of maximizing the model entropy.

    - use_self_normalized_importance_weights – bool if true, use self-normalized importance weighting (normalizing the importance weights such they sum to one), rather than standard importance weighting for estimating the natural gradient.

  _bracketing_search(expected_log_ratios: Tensor, kl_bound: tf.float32, lower_bound: tf.float32, upper_bound: tf.float32) → [tf.float32, tf.float32, <class 'tensorflow.python.framework.ops.Tensor'>]

    This method finds the largest stepsize eta, such that the updated weight distribution stays within a KL-constrained trust-region around the current distribution. Here we use a simple bracketing search, which can be efficiently performed within a Tensorflow graph. The procedure simple keeps track of a lower bound and an upper bound on the optimal stepsize and recursively evaluates the arithmetic mean of both bounds. If this mean-stepsize results in a too large KL divergence, it becomes the new lower bound; otherwise the new upper bound.

    Parameters:

      - expected_log_ratios – tf.Tensor A vector containing an (MC-)estimate of , for every component o.

      - kl_bound – tf.float32 The trust region constraint

      - lower_bound – tf.float32 The initial lower bound on the stepsize

      - upper_bound – The initial upper bound on the stepsize

    Returns:

      new_lower_bound - The lower bound after a stopping criterion was reached.

      new_upper_bound - The upper bound after a stopping criterion was reached.

      new_log_weights - log of the updated weights, .

    Return type:

      tuple(tf.float32, tf.float32, tf.Tensor)

  _update_weights_from_expected_log_ratios(expected_log_ratios, kl_bound)

    Perform the weight update, treating the stepsize as constraint on the KL-divergence.

    Parameters:

      - expected_log_ratios – tf.Variable(tf.float32) A vector containing an (MC-)estimate of , for every component o.

      - stepsize – tf.float32 The stepsize , the new weights will satisfy .

  kl(eta: tf.float32, component_rewards: Tensor) → [tf.float32, <class 'tensorflow.python.framework.ops.Tensor'>]

    Computes the Kullback-Leibler divergence between the updated component and current component, when updating with stepsize eta along the natural gradient.

    Parameters:

      - eta – tf.float32 The stepsize for which the KL divergence should be computed.

      - component_rewards – tf.float32 A tensor containing an MC-estimate of the expected reward (expected logratios) of each component,

    Returns:

      kl - a float corresponding to the KL between the updated and previous weight distribution.

      new_log_weights - log of the updated weights, .

    Return type:

      tuple(float, tf.Tensor)