Bayesian compression for dynamically expandable networks

Yang Yang, Bo Chen, Hongwei Liu

Abstract: Highlights•A compact model structure with preserving the accuracy via sparsity inducing priors, which leads to fewer neurons at each hidden layer in the network, equivalently fewer parameters.•Dynamically expands network capacity with only the necessary number of neurons by employing sparsity inducing priors for the added neurons, so as to increase the network capacity when necessary.•Variational Bayesian approximation for the model parameters with parameter uncertainty.