# Target Distributions

class gmmvi.experiments.target_distributions.lnpdf.LNPDF(use_log_density_and_grad: bool = False, safe_for_tf_graph: bool = True)

  This class defines the interface for target distributions. Every target distribution needs to implement the method log_density(x) for computing the unnormalized log-density of the target distribution.

  This function can be also used to wrap target distributions that are not implemented in Tensorflow, by setting use_log_density_and_grad to True and safe_for_tf_graph to False.

  Parameters:

    - use_log_density_and_grad – bool if False, user is allowed to backprob through self.log_density(), otherwise the method `log_density_and_grad` should be used (and needs to be implemented when using first-order estimates of the NG).

    - safe_for_tf_graph – bool if True, we can call `log_density` and `log_density_and_grad` within a tf.function().

  can_sample() → bool

    If the target distribution can be sampled, and the respective `method` you can overwrite this method to return True.

    Returns:

      is it is safe to call `sample`?

    Return type:

      bool

  expensive_metrics(model: GmmWrapper, samples: Tensor) → dict

    (May not be implemented) This method can be used for computing environment-specific metrics or plots that we want to log. It is called by the `GmmviRunner`.

    Parameters:

      - model – `GmmWrapper` The learned model that we want to evaluate for this target distribution.

      - samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.

    Returns:

      a dictionary containing the name and value for each item we wish to log.

    Return type:

      dict

  get_num_dimensions() → int
    Returns:

      the number of dimensions

    Return type:

      int

  log_density(x: Tensor) → Tensor

    Returns the unnormalized log-density for each sample in x, .

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

    Return type:

      tf.Tensor

  log_density_and_grad(x: Tensor) → [<class 'tensorflow.python.framework.ops.Tensor'>, <class 'tensorflow.python.framework.ops.Tensor'>]

    (May not be implemented) Returns the unnormalized log-density and its gradient for each sample in x.

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      target_log_densities - a one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

      log_density_grads - a two-dimensional tensor of shape number_of_samples x dimensions containing the gradients of the log-densities with respect to the respective sample.

    Return type:

      tuple(tf.Tensor, tf.Tensor)

  property safe_for_tf_graph: bool

    if True, we can call `log_density` and `log_density_and_grad` within a tf.function().

  sample(n: int) → Tensor

    (May not be implemented) If we can sample from the target distribution, this functionality can be implemented here. However, it is not used by the learning algorithm.

    Parameters:

      n – int The number of samples we want to draw

    Returns:

      the sample, a Tensor of shape n x dimensions

    Return type:

      tf.Tensor

  property use_log_density_and_grad: bool

    if False, user is allowed to backprob through self.log_density(), otherwise the method `log_density_and_grad` should be used (and needs to be implemented when using first-order estimates of the NG).

class gmmvi.experiments.target_distributions.gmm.GMM_LNPDF(target_weights: Tensor, target_means: Tensor, target_covs: Tensor)

  Implements a target distribution that is given by a Gaussian mixture model.

  Parameters:

    - target_weights (tf.Tensor of tf.float32) – a one-dimensional vector of size number_of_components containing the mixture weights.

    - target_means (tf.Tensor of tf.float32) – a two-dimensional vector of size number_of_components x dimensions containing the mixture means.

    - target_covs (tf.Tensor of tf.float32) – a three-dimensional vector of size number_of_components x dimensions x dimensions containing the covariance matrices.

  can_sample()
    Returns:

      We can sample from a GMM, so this method will return True.

    Return type:

      bool

  expensive_metrics(model: GmmWrapper, samples: Tensor) → dict

    This method computed the number of detected modes (by testing how many modes of this target distribution are close to a component in the learned model) and a figure that shows plots comparing the marginal distributions of the model with the true marginals of this target distribution.

    Parameters:

      - model – `GmmWrapper` The learned model that we want to evaluate for this target distribution.

      - samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.

    Returns:

      a dictionary containing two items (the number of detected modes, and a figure showing the plots of marginals.

    Return type:

      dict

  get_num_dimensions()
    Returns:

      the number of dimensions

    Return type:

      int

  log_density(x)

    Returns the unnormalized log-density for each sample in x, .

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

    Return type:

      tf.Tensor

  marginal_log_density(x, dim)

    Computes the marginal distribution along the given dimensions.

    Parameters:

      - x (tf.Tensor of tf.float32) – a one-dimensional vector of size number_of_samples containing the samples we want to evaluate

      - dim (an int) – Specifies the dimension used for constructing the marginal GMM.

      - Returns – tf.Tensor - a one-dimensional Tensor of shape number_of_samples containing the marginal log densities.

  sample(n)

    Draws n samples from this GMM.

    Parameters:

      n – int The number of samples we want to draw.

    Returns:

      The sample, a tensor of size n x dimensions.

    Return type:

      tf.Tensor

class gmmvi.experiments.target_distributions.student_t_mixture.StudentTMixture_LNPDF(target_weights: Tensor, target_means: Tensor, target_covs: Tensor, alpha=2)

  Implements a target distribution that is given by a mixture of Student-T distributions.

  Parameters:

    - target_weights (tf.Tensor of tf.float32) – a one-dimensional vector of size number_of_components containing the mixture weights.

    - target_means (tf.Tensor of tf.float32) – a two-dimensional vector of size number_of_components x dimensions containing the mixture means.

    - target_covs (tf.Tensor of tf.float32) – a three-dimensional vector of size number_of_components x dimensions x dimensions containing the covariance matrices.

    - alpha (int) – The number of degrees of freedom.

  can_sample()
    Returns:

      We can sample from a mixture of Student-T, so this method will return True.

    Return type:

      bool

  expensive_metrics(model: GmmWrapper, samples: Tensor) → dict

    This method computed the number of detected modes (by testing how many modes of this mixture of Student-T are close to a component in the learned model) and a figure that shows plots comparing the marginal distributions of the model with the true marginals of this mixture of Student-T.

    Parameters:

      - model – `GmmWrapper` The learned model that we want to evaluate for this target distribution.

      - samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.

    Returns:

      a dictionary containing two items (the number of detected modes, and a figure showing the plots of marginals.

    Return type:

      dict

  get_num_dimensions()
    Returns:

      the number of dimensions

    Return type:

      int

  log_density(x)

    Returns the unnormalized log-density for each sample in x, .

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

    Return type:

      tf.Tensor

  marginal_log_density(x, dim)

    Computes the marginal distribution along the given dimensions.

    Parameters:

      - x – tf.Tensor of tf.float32 a one-dimensional vector of size number_of_samples containing the samples we want to evaluate

      - dim – an int Specifies the dimension used for constructing the marginal mixture of Student-Ts.

    Returns:

      a one-dimensional tensor of shape number_of_samples containing the marginal log densities.

    Return type:

      tf.Tensor

  sample(n)

    Draws n samples from this mixture of Student-T.

    Parameters:

      n – int The number of samples we want to draw.

    Returns:

      The sample, a tensor of size n x dimensions.

    Return type:

      tf.Tensor

class gmmvi.experiments.target_distributions.logistic_regression.LogisticRegression(dataset_id)

  This class is used for implementing the logistic regression experiments based on the BreastCancer and GermanCredit dataset [Lic13], reimplementing the experiments used by Arenz et al. [AZN20].

  Parameters:

    dataset_id – a string Should be either “breast_cancer” or “german_credit”

  get_num_dimensions()
    Returns:

      the number of dimensions

    Return type:

      int

  log_density(x)

    Returns the unnormalized log-density for each sample in x, .

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

    Return type:

      tf.Tensor

  log_likelihood(x)
  property prior_std

class gmmvi.experiments.target_distributions.logistic_regression.LogisticRegression_minibatch(dataset_id, batchsize, size_test_set, use_own_batch_per_sample)

  This class is used for implementing minibatch-variants of the GermanCredit and BreastCancer `experiments`

  Parameters:

    - dataset_id – str Should be either “breast_cancer” or “german_credit”

    - batchsize – int batchsize for evaluating the likelihood.

    - size_test_set – int number of training data that should be held out.

    - use_own_batch_per_samples – bool if True, a different minibatch is used for every sample for which we want to evaluate the target log-density, which reduces the variance (local reparameterization).

  expensive_metrics(model: GmmWrapper, samples: Tensor) → dict

    As target-distribution specific metric, we estimate the full-batch ELBO.

    Parameters:

      - model – `GmmWrapper` The learned model that we want to evaluate for this target distribution.

      - samples – tf.Tensor Samples that have been drawn from the model and that are used for estimating the full-batch ELBO.

    Returns:

      a dictionary with a single item containing the full-batch elbo.

    Return type:

      dict

  likelihood_batch(x, data, labels)
  log_density(x)

    Returns the unnormalized log-density for each sample in x, .

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

    Return type:

      tf.Tensor

  log_density_fb(x)

    Evaluate the log-density on the full data set (used for evaluation). If size_test_set=0, this function is equivalent to `gmmvi.experiments.target_distributions.logistic_regression.LogisticRegression.log_density()`.

  shuffle_data()

class gmmvi.experiments.target_distributions.planar_robot.PlanarRobot(num_links, num_goals, prior_std=0.2, likelihood_std=0.01)

  This class reimplements the “PlanarRobot” experiments used by Arenz et al. [AZN20].

  Parameters:

    - num_links – int The number of links of the robot

    - num_goals – int The number of goal positions, must be either 1 or 4

    - prior_std – float The standard deviation of the (zero-mean) prior on the joint angles. The first value is ignored, as the first link always has a standard deviation of 1.

    - likelihood_std – float The std-deviation used for penalizing the distance in X-Y between the robot endeffective and the goal position.

  expensive_metrics(model: GmmWrapper, samples: Tensor) → dict

    This method computes two task-specific metrics:

    1. The number of detected modes: This is course heuristic to count the different configurations used for reaching each of the goal positions (potentially misleading!)

    2. Plots of the mean configurations of the learned model

    Parameters:

      - model – `GmmWrapper` The learned model that we want to evaluate for this target distribution.

      - samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.

    Returns:

      a dictionary containing two items (the number of detected modes, and a figure showing the mean configurations).

    Return type:

      dict

  forward_kinematics(theta)
  get_num_dimensions()
    Returns:

      the number of dimensions

    Return type:

      int

  likelihood(pos: Tensor) → Tensor
  log_density(theta)

    Returns the unnormalized log-density for each sample in x, .

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

    Return type:

      tf.Tensor

class gmmvi.experiments.target_distributions.bnn.BNN_LNPDF(likelihood_scaling, dataset_seed, prior_std, batch_size, hidden_units, loss, activations)

  This class is used for implementing the target distribution given by the posterior for a Bayesian Neural Network.

  Parameters:

    - likelihood_scaling – float a coefficient that can be used to scale the effect of the likelihood

    - dataset_seed – int The dataset_seed is used for reproducible train/test-splits

    - prior_std – float The standard deviation of the (zero-mean) prior over the network weights

    - batch_size – int size of the minibatches

    - hidden_units – list[int] The length of the list defines the number of hidden layers, the entries define their width

    - loss – a tf.Keras.losses The loss function used for computing the log-likelihood

    - activations – a list of Tensorflow activation functions activations for each hidden layer and the output layer

  average_loss(x, dataset)
  avg_bayesian_inference_loss(x, dataset)
  avg_bayesian_inference_test_loss(x, num_batches)
  bayesian_inference_test_loss(x)
  create_model()
  forward_from_weight_vector(input, x)
  get_num_dimensions()
    Returns:

      the number of dimensions

    Return type:

      int

  log_density(x)

    Returns the unnormalized log-density for each sample in x, .

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

    Return type:

      tf.Tensor

  log_density_and_grad(x: Tensor) → [<class 'tensorflow.python.framework.ops.Tensor'>, <class 'tensorflow.python.framework.ops.Tensor'>]

    (May not be implemented) Returns the unnormalized log-density and its gradient for each sample in x.

    Parameters:

      x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.

    Returns:

      target_log_densities - a one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.

      log_density_grads - a two-dimensional tensor of shape number_of_samples x dimensions containing the gradients of the log-densities with respect to the respective sample.

    Return type:

      tuple(tf.Tensor, tf.Tensor)

  log_likelihood(x)
  log_likelihood_and_grad(x)
  log_likelihood_old(x)
  log_prior(x, ignore_constant=False)
  log_prior_and_grad(x, ignore_constant=False)
  prepare_data()
  property prior_std