Deep Linear Networks with Arbitrary Loss: All Local Minima Are Global

Thomas Laurent, James von Brecht

Abstract: We consider deep linear networks with arbitrary convex differentiable loss. We provide a short and elementary proof of the fact that all local minima are global minima if the hidden layers are eith...