Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models

Shinnosuke Saito, Takashi Matsubara

Keywords: diffusion models, manifold hypothesis, Riemannian geometry, image interpolation

TL;DR: We define a Riemannian metric on the noise space of diffusion models using the Jacobian of score function. Under this metric, geodesics stay within or run parallel to the data manifold, producing more perceptually natural and faithful transitions.

Abstract: Diffusion models are powerful deep generative models (DGMs) that generate high-fidelity, diverse content. However, unlike classical DGMs, they lack an explicit, tractable low-dimensional latent space that parameterizes the data manifold. This absence limits manifold-aware analysis and operations, such as interpolation and editing. Existing interpolation methods for diffusion models typically follow paths through high-density regions, which are not necessarily aligned with the data manifold and can yield perceptually unnatural transitions. To exploit the data manifold learned by diffusion models, we propose a novel Riemannian metric on the noise space, inspired by recent findings that the Jacobian of the score function captures the tangent spaces to the local data manifold. This metric encourages geodesics in the noise space to stay within or run parallel to the learned data manifold. Experiments on image interpolation show that our metric produces perceptually more natural and faithful transitions than existing density-based and naive baselines.

Primary Area: generative models

Submission Number: 10867