Unitary Evolution Recurrent Neural Networks

Martín Arjovsky, Amar Shah, Yoshua Bengio

Abstract: Recurrent neural networks (RNNs) are notoriously difficult to train. When the eigenvalues of the hidden to hidden weight matrix deviate from absolute value 1, optimization becomes difficult due to ...