Efficient Network Structure via Ensemble Deep Evolutionary Synthesis

Keyvan Kasiri, Mohammad Javad Shafiee, Alexander Wong, Justin Eichel

Abstract:

While deep neural networks have shown promising results in a wide range of
applications on highly powerful computational devices, one challenging task is
to deploy a deep neural network on embedded devices for the widespread use.
Deep neural networks and specially convolutional neural networks are usually
over-parameterized and one possible solution is to remodel the network architecture
with a smaller network architecture with a trade-off on modeling accuracy
and performance. Here we take advantage of meta-learning algorithms to synthesize
a more efficient model while it boosts the modeling performance. To this
end, we propose an ensemble of deep evolutionary intelligence frameworks where
it synthesizes several very efficient models with less than 3% drop on modeling
accuracy and then aggregates them to boost the modeling performance. Experimental
results demonstrates that the proposed ensemble of Deep Evolutionary
Synthesis approach synthesizes an ensemble model which is 1.5X smaller than
the original network architecture while performing more accurate (83.30% compared
to 83.18%) than the original network in terms of modeling accuracy for
binary object segmentation.

Keywords: Deep neural networks, evolutionary synthesis, ensemble learning