Zero Generalization Error Theorem for Random Interpolators via Algebraic Geometry

ICLR 2026 Conference Submission18571 Authors

Keywords: generalization error, interpolator, algebraic geometry

Abstract: We theoretically demonstrate that the generalization error of interpolators for general machine learning models becomes $0$ once the number of training samples exceeds a certain threshold. Understanding the high generalization ability of large-scale models such as deep neural networks (DNNs) remains one of the central open problems in machine learning theory. While recent theoretical studies have attributed this phenomenon to the implicit bias of stochastic gradient descent (SGD) toward well-generalizing solutions, empirical evidences indicate that it primarily stems from properties of the model itself. Specifically, even randomly sampled interpolators—parameters that achieve zero training error—have been observed to generalize effectively. In this study, under a teacher–student framework, we prove that the generalization error of randomly sampled interpolators becomes exactly zero once the number of training samples exceeds a threshold determined by the geometric structure of the interpolator set in parameter space. As a proof technique, we leverage tools from algebraic geometry to mathematically characterize this geometric structure.

Primary Area: learning theory

Submission Number: 18571