Efficient Network Structure via Ensemble Deep Evolutionary Synthesis

Keyvan Kasiri, Mohammad Javad Shafiee, Alexander Wong, Justin Eichel

Feb 12, 2018 (modified: Feb 12, 2018) ICLR 2018 Workshop Submission readers: everyone
  • Abstract: While deep neural networks have shown promising results in a wide range of applications on highly powerful computational devices, one challenging task is to deploy a deep neural network on embedded devices for the widespread use. Deep neural networks and specially convolutional neural networks are usually over-parameterized and one possible solution is to remodel the network architecture with a smaller network architecture with a trade-off on modeling accuracy and performance. Here we take advantage of meta-learning algorithms to synthesize a more efficient model while it boosts the modeling performance. To this end, we propose an ensemble of deep evolutionary intelligence frameworks where it synthesizes several very efficient models with less than 3% drop on modeling accuracy and then aggregates them to boost the modeling performance. Experimental results demonstrates that the proposed ensemble of Deep Evolutionary Synthesis approach synthesizes an ensemble model which is 1.5X smaller than the original network architecture while performing more accurate (83.30% compared to 83.18%) than the original network in terms of modeling accuracy for binary object segmentation.
  • Keywords: Deep neural networks, evolutionary synthesis, ensemble learning