Can Implicit Bias Explain Generalization? Stochastic Convex Optimization as a Case Study

Assaf Dauber, Meir Feder, Tomer Koren, Roi Livni

Abstract: The notion of implicit bias, or implicit regularization, has been suggested as a means to explain the surprising generalization ability of modern-days overparameterized learning algorithms. This notion refers to the tendency of the optimization algorithm towards a certain structured solution that often generalizes well. Recently, several papers have studied implicit regularization and were able to identify this phenomenon in various scenarios.