Negative eigenvalues of the Hessian in deep neural networksDownload PDF

Feb 12, 2018 (edited Jun 04, 2018)ICLR 2018 Workshop SubmissionReaders: Everyone
  • Keywords: Optimization, Hessian matrix, Neural Networks, Negative Curvature
  • TL;DR: We study negative curvature of the loss of neural networks. We want to develop a better optimization method.
  • Abstract: We study the loss function of a deep neural network through the eigendecomposition of its Hessian matrix. We focus on negative eigenvalues, how important they are, and how to best deal with them. The goal is to develop an optimization method specifically tailored for deep neural networks.
3 Replies