To train the sigma-VAE run and reproduce the results in Table 1:

# Sigma-VAE
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/sigma --sigma_mode scalar_fixed
# Gaussian VAE
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/gaussian --sigma_mode scalar_fixed --learn_beta=0
# Optimal sigma-VAE
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/optimal_constant --sigma_mode optimal_constant
# Optimal per-image sigma-VAE
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/optimal_scalar --sigma_mode optimal
# Beta distribution
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/betad --sigma_mode scalar_fixed --distribution beta

Discrete distributions:
# Discrete logistic mixture
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/lb_image_modlog --sigma_mode image --distribution=discrete_logistic_mixture
# Categorical
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/categorical --distribution=categorical
# Bernoulli
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/bernoulli --distribution=bernoulli
# Bitwise-categorical
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/bwcategorical --distribution=bitwise_categorical

beta-VAE:
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/beta0001 --sigma_mode scalar_fixed --learn_beta=0 --sigma=0.0223
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/beta001 --sigma_mode scalar_fixed --learn_beta=0 --sigma=0.0707
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/beta01 --sigma_mode scalar_fixed --learn_beta=0 --sigma=0.223
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/beta1 --sigma_mode scalar_fixed --learn_beta=0 --sigma=0.707
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/beta10 --sigma_mode scalar_fixed --learn_beta=0 --sigma=2.23
python3 -m pdb -c continue train_vae.py --dataset=SVHN --log_dir svhn/beta100 --sigma_mode scalar_fixed --learn_beta=0 --sigma=7.07


For the marginal KL computation experiment, see the file:
InspectKL.ipynb

Also find pre-trained models in the directory:
pretrained_models