Target Distributions¶

- class gmmvi.experiments.target_distributions.lnpdf.LNPDF(use_log_density_and_grad: bool = False, safe_for_tf_graph: bool = True)[source]¶
This class defines the interface for target distributions. Every target distribution needs to implement the method log_density(x) for computing the unnormalized log-density of the target distribution.
This function can be also used to wrap target distributions that are not implemented in Tensorflow, by setting use_log_density_and_grad to True and safe_for_tf_graph to False.
- Parameters:
use_log_density_and_grad – bool if False, user is allowed to backprob through self.log_density(), otherwise the method
log_density_and_grad
should be used (and needs to be implemented when using first-order estimates of the NG).safe_for_tf_graph – bool if True, we can call
log_density
andlog_density_and_grad
within a tf.function().
- can_sample() bool [source]¶
If the target distribution can be sampled, and the respective
method
you can overwrite this method to return True.
- expensive_metrics(model: GmmWrapper, samples: Tensor) dict [source]¶
(May not be implemented) This method can be used for computing environment-specific metrics or plots that we want to log. It is called by the
GmmviRunner
.- Parameters:
model –
GmmWrapper
The learned model that we want to evaluate for this target distribution.samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.
- Returns:
a dictionary containing the name and value for each item we wish to log.
- Return type:
- log_density(x: Tensor) Tensor [source]¶
Returns the unnormalized log-density for each sample in x,
.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
- Return type:
tf.Tensor
- log_density_and_grad(x: Tensor) [<class 'tensorflow.python.framework.ops.Tensor'>, <class 'tensorflow.python.framework.ops.Tensor'>] [source]¶
(May not be implemented) Returns the unnormalized log-density and its gradient for each sample in x.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
target_log_densities - a one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
log_density_grads - a two-dimensional tensor of shape number_of_samples x dimensions containing the gradients of the log-densities with respect to the respective sample.
- Return type:
tuple(tf.Tensor, tf.Tensor)
- property safe_for_tf_graph: bool¶
if True, we can call
log_density
andlog_density_and_grad
within a tf.function().
- sample(n: int) Tensor [source]¶
(May not be implemented) If we can sample from the target distribution, this functionality can be implemented here. However, it is not used by the learning algorithm.
- Parameters:
n – int The number of samples we want to draw
- Returns:
the sample, a Tensor of shape n x dimensions
- Return type:
tf.Tensor
- property use_log_density_and_grad: bool¶
if False, user is allowed to backprob through self.log_density(), otherwise the method
log_density_and_grad
should be used (and needs to be implemented when using first-order estimates of the NG).
- class gmmvi.experiments.target_distributions.gmm.GMM_LNPDF(target_weights: Tensor, target_means: Tensor, target_covs: Tensor)[source]¶
Implements a target distribution that is given by a Gaussian mixture model.
- Parameters:
target_weights (tf.Tensor of tf.float32) – a one-dimensional vector of size number_of_components containing the mixture weights.
target_means (tf.Tensor of tf.float32) – a two-dimensional vector of size number_of_components x dimensions containing the mixture means.
target_covs (tf.Tensor of tf.float32) – a three-dimensional vector of size number_of_components x dimensions x dimensions containing the covariance matrices.
- can_sample()[source]¶
- Returns:
We can sample from a GMM, so this method will return True.
- Return type:
- expensive_metrics(model: GmmWrapper, samples: Tensor) dict [source]¶
This method computed the number of detected modes (by testing how many modes of this target distribution are close to a component in the learned model) and a figure that shows plots comparing the marginal distributions of the model with the true marginals of this target distribution.
- Parameters:
model –
GmmWrapper
The learned model that we want to evaluate for this target distribution.samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.
- Returns:
a dictionary containing two items (the number of detected modes, and a figure showing the plots of marginals.
- Return type:
- log_density(x)[source]¶
Returns the unnormalized log-density for each sample in x,
.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
- Return type:
tf.Tensor
- marginal_log_density(x, dim)[source]¶
Computes the marginal distribution along the given dimensions.
- Parameters:
x (tf.Tensor of tf.float32) – a one-dimensional vector of size number_of_samples containing the samples we want to evaluate
dim (an int) – Specifies the dimension used for constructing the marginal GMM.
Returns – tf.Tensor - a one-dimensional Tensor of shape number_of_samples containing the marginal log densities.
- class gmmvi.experiments.target_distributions.student_t_mixture.StudentTMixture_LNPDF(target_weights: Tensor, target_means: Tensor, target_covs: Tensor, alpha=2)[source]¶
Implements a target distribution that is given by a mixture of Student-T distributions.
- Parameters:
target_weights (tf.Tensor of tf.float32) – a one-dimensional vector of size number_of_components containing the mixture weights.
target_means (tf.Tensor of tf.float32) – a two-dimensional vector of size number_of_components x dimensions containing the mixture means.
target_covs (tf.Tensor of tf.float32) – a three-dimensional vector of size number_of_components x dimensions x dimensions containing the covariance matrices.
alpha (int) – The number of degrees of freedom.
- can_sample()[source]¶
- Returns:
We can sample from a mixture of Student-T, so this method will return True.
- Return type:
- expensive_metrics(model: GmmWrapper, samples: Tensor) dict [source]¶
This method computed the number of detected modes (by testing how many modes of this mixture of Student-T are close to a component in the learned model) and a figure that shows plots comparing the marginal distributions of the model with the true marginals of this mixture of Student-T.
- Parameters:
model –
GmmWrapper
The learned model that we want to evaluate for this target distribution.samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.
- Returns:
a dictionary containing two items (the number of detected modes, and a figure showing the plots of marginals.
- Return type:
- log_density(x)[source]¶
Returns the unnormalized log-density for each sample in x,
.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
- Return type:
tf.Tensor
- marginal_log_density(x, dim)[source]¶
Computes the marginal distribution along the given dimensions.
- Parameters:
x – tf.Tensor of tf.float32 a one-dimensional vector of size number_of_samples containing the samples we want to evaluate
dim – an int Specifies the dimension used for constructing the marginal mixture of Student-Ts.
- Returns:
a one-dimensional tensor of shape number_of_samples containing the marginal log densities.
- Return type:
tf.Tensor
- class gmmvi.experiments.target_distributions.logistic_regression.LogisticRegression(dataset_id)[source]¶
This class is used for implementing the logistic regression experiments based on the BreastCancer and GermanCredit dataset [Lic13], reimplementing the experiments used by Arenz et al. [AZN20].
- Parameters:
dataset_id – a string Should be either “breast_cancer” or “german_credit”
- log_density(x)[source]¶
Returns the unnormalized log-density for each sample in x,
.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
- Return type:
tf.Tensor
- property prior_std¶
- class gmmvi.experiments.target_distributions.logistic_regression.LogisticRegression_minibatch(dataset_id, batchsize, size_test_set, use_own_batch_per_sample)[source]¶
This class is used for implementing minibatch-variants of the GermanCredit and BreastCancer
experiments
- Parameters:
dataset_id – str Should be either “breast_cancer” or “german_credit”
batchsize – int batchsize for evaluating the likelihood.
size_test_set – int number of training data that should be held out.
use_own_batch_per_samples – bool if True, a different minibatch is used for every sample for which we want to evaluate the target log-density, which reduces the variance (local reparameterization).
- expensive_metrics(model: GmmWrapper, samples: Tensor) dict [source]¶
As target-distribution specific metric, we estimate the full-batch ELBO.
- Parameters:
model –
GmmWrapper
The learned model that we want to evaluate for this target distribution.samples – tf.Tensor Samples that have been drawn from the model and that are used for estimating the full-batch ELBO.
- Returns:
a dictionary with a single item containing the full-batch elbo.
- Return type:
- log_density(x)[source]¶
Returns the unnormalized log-density for each sample in x,
.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
- Return type:
tf.Tensor
- log_density_fb(x)[source]¶
Evaluate the log-density on the full data set (used for evaluation). If size_test_set=0, this function is equivalent to
gmmvi.experiments.target_distributions.logistic_regression.LogisticRegression.log_density()
.
- class gmmvi.experiments.target_distributions.planar_robot.PlanarRobot(num_links, num_goals, prior_std=0.2, likelihood_std=0.01)[source]¶
This class reimplements the “PlanarRobot” experiments used by Arenz et al. [AZN20].
- Parameters:
num_links – int The number of links of the robot
num_goals – int The number of goal positions, must be either 1 or 4
prior_std – float The standard deviation of the (zero-mean) prior on the joint angles. The first value is ignored, as the first link always has a standard deviation of 1.
likelihood_std – float The std-deviation used for penalizing the distance in X-Y between the robot endeffective and the goal position.
- expensive_metrics(model: GmmWrapper, samples: Tensor) dict [source]¶
This method computes two task-specific metrics:
The number of detected modes: This is course heuristic to count the different configurations used for reaching each of the goal positions (potentially misleading!)
Plots of the mean configurations of the learned model
- Parameters:
model –
GmmWrapper
The learned model that we want to evaluate for this target distribution.samples – tf.Tensor Samples that have been drawn from the model, which can be used for evaluations.
- Returns:
a dictionary containing two items (the number of detected modes, and a figure showing the mean configurations).
- Return type:
- log_density(theta)[source]¶
Returns the unnormalized log-density for each sample in x,
.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
- Return type:
tf.Tensor
- class gmmvi.experiments.target_distributions.bnn.BNN_LNPDF(likelihood_scaling, dataset_seed, prior_std, batch_size, hidden_units, loss, activations)[source]¶
This class is used for implementing the target distribution given by the posterior for a Bayesian Neural Network.
- Parameters:
likelihood_scaling – float a coefficient that can be used to scale the effect of the likelihood
dataset_seed – int The dataset_seed is used for reproducible train/test-splits
prior_std – float The standard deviation of the (zero-mean) prior over the network weights
batch_size – int size of the minibatches
hidden_units – list[int] The length of the list defines the number of hidden layers, the entries define their width
loss – a tf.Keras.losses The loss function used for computing the log-likelihood
activations – a list of Tensorflow activation functions activations for each hidden layer and the output layer
- log_density(x)[source]¶
Returns the unnormalized log-density for each sample in x,
.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
A one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
- Return type:
tf.Tensor
- log_density_and_grad(x: Tensor) [<class 'tensorflow.python.framework.ops.Tensor'>, <class 'tensorflow.python.framework.ops.Tensor'>] [source]¶
(May not be implemented) Returns the unnormalized log-density and its gradient for each sample in x.
- Parameters:
x – tf.Tensor The samples that we want to evaluate, a tf.Tensor of shape number_of_samples x dimensions.
- Returns:
target_log_densities - a one-dimensional tensor of shape number_of_samples containing the unnormalized log-densities.
log_density_grads - a two-dimensional tensor of shape number_of_samples x dimensions containing the gradients of the log-densities with respect to the respective sample.
- Return type:
tuple(tf.Tensor, tf.Tensor)
- property prior_std¶