Score-Based Deterministic Density Sampling

Vasily Ilin, Bamdad Hosseini, Jingwei Hu

Keywords: sampling, score, neural network, Wasserstein gradient flow, relative entropy

TL;DR: Deterministic sampling of an unnormalized density using on-the-fly score estimation with a neural network.

Abstract: We propose and analyze a deterministic sampling framework using Score-Based Transport Modeling (SBTM) for sampling an unnormalized target density $\pi$. While diffusion generative modeling relies on pre-training the score function $\nabla \log f_t$ using samples from $\pi$, SBTM addresses the more general and challenging setting where only the $\nabla \log\pi$ is known. SBTM approximates the Wasserstein gradient flow on $\mathrm{KL}(f_t\|\pi)$ by learning the time-varying score $\nabla \log f_t$ on the fly using score matching. The learned score gives immediate access to relative Fisher information, providing a built‑in convergence diagnostic. The deterministic trajectories are smooth, interpretable, and free of Brownian-motion noise, while having the same distribution as ULA. We prove that SBTM dissipates relative entropy at the same rate as the exact gradient flow, provided sufficient training. We further extend our framework to annealed dynamics, to handle non log-concave targets. Numerical experiments validate our theoretical findings: SBTM converges at the optimal rate, has smooth trajectories, and is easily integrated with annealed dynamics. We compare to the baselines of ULA and annealed ULA.

Submission Number: 83