Dynamics of Training

Siegfried Bös, Manfred Opper

Abstract: A new method to calculate the full training process of a neural net(cid:173) work is introduced. No sophisticated methods like the replica trick are used. The results are directly related to the actual number of training steps. Some results are presented here, like the maximal learning rate, an exact description of early stopping, and the neces(cid:173) sary number of training steps. Further problems can be addressed with this approach.