AutoML to generate ensembles of deep neural networks

Pierrick Pochelu, Serge G. Petiton, Bruno Conche

Keywords: Deep Learning, Ensemble, AutoML

Abstract:

Automated Machine Learning with ensembling seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions.
AutoML and Ensemble of Deep Neural Network produce qualitative results but they are computing intensive methods in both building and inference run time. Therefore, an ideal method would produce at one AutoML run time different ensembles regarding accuracy and inference speed regarding the desired trade-off.
Despite multiple initiative for non-deep machine learning have been proposed there still no consensus on how to automatically construct efficient ensembles of deep neural networks.
First, we propose a new multi-objective ensemble selection method to generate efficient ensembles by controlling their computing cost named SMOBF. Second, we propose an AutoML workflow using Hyperband to generate DNNs, SMOBF to combine DNNs and the simple averaging as combination rule. Finally we compare this AutoML workflow to several baselines and its inherent characteristics are discussed.
It shows robust results leveraging multiple GPUs on two datasets but can be applied beyond.

One-sentence Summary: We propose an efficient and flexible AutoML workflow to generate an ensemble of deep neural networks.