On Connected Sublevel Sets in Deep Learning

Quynh Nguyen

Abstract: This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies t...