Reliably Learning the ReLU in Polynomial Time

Surbhi Goel, Varun Kanade, Adam R. Klivans, Justin Thaler

Abstract: We give the first dimension-efficient algorithms for learning Rectified Linear Units (ReLUs), which are functions of the form $\mathbf{x} \mapsto \mathsf{max}(0,  \mathbf{w} ⋅\mathbf{x})$ with $\ma...