Advanced Neuroevolution: A gradient-free algorithm to train Deep Neural Networks

Ahmed Aly, David Weikersdorfer, Claire Delaunay

Abstract: In this paper we present a novel optimization algorithm called Advanced Neuroevolution. The aim for this algorithm is to train deep neural networks, and eventually act as an alternative to Stochastic Gradient Descent (SGD) and its variants as needed.We evaluated our algorithm on the MNIST dataset, as well as on several global optimization problems such as the Ackley function. We find the algorithm performing relatively well for both cases, overtaking other global optimization algorithms such as Particle Swarm Optimization (PSO) and Evolution Strategies (ES).

Keywords: Evolutionary Algorithm, Optimization, MNIST

TL;DR: A new algorithm to train deep neural networks. Tested on optimization functions and MNIST.