Denoising for Manifold Extrapolation

Zeyu Yun, Galen Chuang, Derek Dong, Yubei Chen

TL;DR: You can use a denoiser to do image extrapolation.

Abstract:

Deep neural network-based image denoisers are the key component in high quality diffusion models. Unlike latent variable models, denoisers are not explicitly trained to extract latent manifolds. However, recent work suggests that denoisers implicitly capture the geometry of these manifolds. Given that images lie on a low-dimensional latent manifold embedded in a high dimensional space, this raises the question: can we recover a latent manifold from a diffusion model? Here, we demonstrate that a manifold embedded in a trained denoiser can be extracted and visualized through manifold extrapolation.
We provide a theoretical framework and an algorithm for performing manifold extrapolation using a denoiser, and we show that our approach outperforms traditional manifold extrapolation methods. Finally, we demonstrate that when the latent manifold is simple and low-dimensional, it can be extracted using manifold extrapolation with only two nearby points.

Style Files: I have used the style files.

Submission Number: 76