Exponentially many local minima for single neurons

Peter Auer, Mark Herbster, Manfred K. Warmuth

Abstract: We show that for a single neuron with the logistic function as the transfer function the number of local minima of the error function based on the square loss can grow exponentially in the dimension.