WeightStepsizeAdaptation¶

- class gmmvi.optimization.gmmvi_modules.weight_stepsize_adaptation.WeightStepsizeAdaptation(initial_stepsize: tf.float32)[source]¶
This class provides a common interface for adapting the stepsize for the weight update.
There are currently three options for weight stepsize adpatation:
The
FixedWeightStepsizeAdaptation
is a dummy-class, that does not do anything.The
DecayingWeightStepsizeAdaptation
uses exponential decay.The
ImprovementBasedWeightStepsizeAdaptation
uses a procedure similar to VIPS [AZN20] to increase the stepsize if the mixture improved during the last updates, and to decrease it otherwise.
- Parameters:
initial_stepsize – float The initial stepsize for the weight update.
- static build_from_config(config, gmm_wrapper)[source]¶
This static method provides a convenient way to create a
FixedWeightStepsizeAdaptation
,DecayingWeightStepsizeAdaptation
orImprovementBasedWeightStepsizeAdaptation
depending on the provided config.- Parameters:
config – dict The dictionary is typically read from YAML a file, and holds all hyperparameters.
gmm_wrapper –
GmmWrapper
The wrapped model.
DecayingWeightStepsizeAdaptation¶
- class gmmvi.optimization.gmmvi_modules.weight_stepsize_adaptation.DecayingWeightStepsizeAdaptation(initial_stepsize: tf.float32, annealing_exponent: tf.float32)[source]¶
This class implements an exponentially decaying stepsize schedule.
See [KNT+18].
- Parameters:
gmm_wrapper –
GmmWrapper
The wrapped.annealing_exponent – float controls how fast the stepsize decays
initial_stepsize – float The initial stepsize for the weight update.
FixedWeightStepsizeAdaptation¶
- class gmmvi.optimization.gmmvi_modules.weight_stepsize_adaptation.FixedWeightStepsizeAdaptation(initial_stepsize: tf.float32)[source]¶
This class is a dummy class, that can be used when we want to keep the stepsize for the weight update constant.
- Parameters:
initial_stepsize – float The initial stepsize for the weight update.
ImprovementBasedWeightStepsizeAdaptation¶
- class gmmvi.optimization.gmmvi_modules.weight_stepsize_adaptation.ImprovementBasedWeightStepsizeAdaptation(model: GmmWrapper, initial_stepsize: tf.float32, min_stepsize: tf.float32, max_stepsize: tf.float32, stepsize_inc_factor: tf.float32, stepsize_dec_factor: tf.float32)[source]¶
Increases the stepsize if the last weight update increased its reward, decreases it otherwise.
- Parameters:
gmm_wrapper –
GmmWrapper
The wrapped model.initial_stepsize – float The initial stepsize for the weight update.
min_stepsize – float Do not not decrease the stepsize below this point
max_stepsize – float Do not increase the stepsize above this point
stepsize_inc_factor – float Factor (>1) for increasing the stepsize
stepsize_dec_factor – float Factor in the range [0, 1] for decreasing the stepsize