A Mini-Block Fisher Method for Deep Neural Networks

Achraf Bahamou, Donald Goldfarb, Yi Ren

Abstract: Deep Neural Networks (DNNs) are currently predominantly trained using first-order methods. Some of these methods (e.g., Adam, AdaGrad, and RMSprop, and their variants) incorporate a small amount of...