Multiple Gain Adaptations for Improved Neural Networks Training

Jeshwanth Challagundla, Kanishka Tyagi, Tushar Chugh, Michael Manry

Abstract: A two stage algorithm developed called Multiple Gain Adaptations for Improved Networks (MGAIN) is presented. MAGAIN alternatively finds output weights and uses several gain factors to update the input weights in a Multi-Layer Perceptron. The gain factors are computed using Newtons method. Our method dynamically adjusts the quantity of gain factors calculated to maximize the reduction in loss with each epoch. The results demonstrate that our approach outperforms existing second order algorithms across the majority of diverse datasets.