Log-density Hessian estimation without the curse of dimensionality via denoising score matching

Konstantin Yakovlev, Anna Markovich, Nikita Puchkin

Keywords: generative diffusion models, Fisher divergence, Gagliardo-Nirenberg inequality, score Jacobian matrix estimation

TL;DR: Denoising score matching, assuming that the data distribution exhibits a low-dimensional structure, yields a score estimate whose differentiation recovers the log‑density Hessian without the curse of dimensionality..

Abstract:

We study the problem of estimating the score function and its Jacobian matrix using denoising score matching.
Assuming that the data distribution exhibits a low-dimensional structure, we prove that denoising score matching is able to estimate log-density Hessian without the curse of dimensionality by simple differentiation.
This justifies convergence of ODE-based samplers for generative diffusion models.
Our approach is based on Gagliardo-Nirenberg-type inequalities relating weighted $L^2$-norms of smooth functions and their derivatives.

Submission Number: 93