On the loss landscape of a class of deep neural networks with no bad local valleysDownload PDF

27 Sep 2018 (modified: 24 Dec 2018)ICLR 2019 Conference Blind SubmissionReaders: Everyone
  • Abstract: We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily close to zero. This implies that these networks have no sub-optimal strict local minima.
  • Keywords: loss landscape, local minima, deep neural networks
14 Replies

Loading