On the Impact of the Activation function on Deep Neural Networks Training

Soufiane Hayou, Arnaud Doucet, Judith Rousseau

Abstract: The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of ...