A New Perspective on Minimum-Norm Interpolation Under Gaussian Covariates

Gil Kur, Zong Shang, Paul Simanjuntak, Guillaume Lecué, Reese Pathak

Abstract: Minimum-Norm Interpolators (MNI) in overparameterized linear models have gained attention as a tractable framework for studying interpolation phenomena that resemble empirical observations in neural networks. Most prior work on these interpolators either exploits closed-form solutions when available or relies heavily on Gaussian comparison results, such as the convex Gaussian Min-Max Theorem (CGMT). In this paper, we introduce a new perspective on MNI under isotropic Gaussian covariates by leveraging tools from high-dimensional geometry. First, we obtain a ``localized'' bound on the MNI's shrinkage of the original ground truth that occurs under isotropic Gaussian covariates when the norm is in an isotropic position. Then, we prove a sharp bound on the Mean Squared Error (MSE) of the $\ell_1$-MNI, as obtained by Wang 22' via a geometric proof, which avoids invoking the CGMT and instead relies on the work of Fleury 12' on Gaussian polytopes.

Code Dataset Promise: No

Format Confirmation: I agree that I have read and followed the formatting instructions for the camera ready version.

Submission Number: 1989