Abstract: We consider nonlinear networks as perturbations of linear ones. Based on this approach, we present a novel generalization bound that become non-vacuous for networks that are close to being linear. The main advantage over the previous works which propose non-vacuous generalization bounds is that our bound is *a priori*: performing the actual training is not required for evaluating the bound. To the best of our knowledge, it is the first non-vacuous generalization bound for neural nets possessing this property.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: Added a section "Our approach" in the main, explaining the high-level approach we take in our work. Also, multiple minor modifications (grammar, phrasing, clarifications).
Supplementary Material: zip
Assigned Action Editor: ~Jeffrey_Pennington1
Submission Number: 2981
Loading