Ensemble Distribution Distillation

Andrey Malinin, Bruno Mlodozeniec, Mark J. F. Gales

Abstract: We distill an ensemble of models into a single model, capturing both the improved classification performance and information about the diversity of the ensemble, which is useful for uncertainty estimation.