Adaptive momentum with discriminative weight for neural network stochastic optimization

Jiyang Bai, Yuxiang Ren, Jiawei Zhang

Abstract: Optimization algorithms with momentum have been widely used for building deep learning models because of the fast convergence rate. Momentum helps accelerate Stochastic gradient descent in relevant d...