Barcodes as summary of objective functions' topologyDownload PDF

25 Sep 2019 (modified: 24 Dec 2019)ICLR 2020 Conference Blind SubmissionReaders: Everyone
  • Original Pdf: pdf
  • TL;DR: We apply canonical forms of gradient complexes (barcodes) to explore neural networks loss surfaces.
  • Abstract: We apply canonical forms of gradient complexes (barcodes) to explore neural networks loss surfaces. We present an algorithm for calculations of the objective function's barcodes of minima. Our experiments confirm two principal observations: (1) the barcodes of minima are located in a small lower part of the range of values of objective function and (2) increase of the neural network's depth brings down the minima's barcodes. This has natural implications for the neural network learning and the ability to generalize.
  • Keywords: Barcodes, canonical form invariants, loss surface, gradient complexes
7 Replies

Loading