On the Interpolation Effect of Score Smoothing in Diffusion Models

ICLR 2026 Conference Submission20023 Authors

Keywords: score-based diffusion model, empirical score function, norm-bounded neural network, barron norm, function smoothing, subspace recovery

Abstract: Score-based diffusion models have achieved remarkable progress in various domains with the ability to generate new data samples that do not exist in the training set. In this work, we study the hypothesis that such creativity arises from an interpolation effect caused by a smoothing of the empirical score function. Focusing on settings where the training set lies uniformly in a one-dimensional subspace, we probe the interplay between score smoothing and the denoising dynamics with analytical solutions and numerical experiments. In particular, we demonstrate how a smoothed score function can lead to the generation of samples that interpolate among the training data within their subspace while avoiding full memorization. Moreover, we present theoretical and empirical evidence that learning score functions with regularized neural networks can have a similar effect as score smoothing, including in simple nonlinear settings.

Primary Area: learning theory

Submission Number: 20023