Analytical solutions for a family of single layer neural network regression problems

Siddharth Krishna Kumar

Keywords: deep learning, analytical solution

TL;DR: The optimal weights for a family of penalized single layer neural network regression problems wherein the response variable has all non-positive entries will always be 0

Abstract: In this paper, we analyze a family of penalized single layer neural network regression problems wherein the response variable has all non-negative entries. We show analytically that the optimal weights of the problem lie at the vector of zeros, which is a point of non-differentiability.