Diffusion Based Representation Learning
Abstract: Diffusion-based methods, represented as stochastic differential equations on a continuous-time domain, have recently proven successful as non-adversarial generative models. Training such models relies on denoising score matching, which can be seen as multi-scale denoising autoencoders. Here, we augment the denoising score matching framework to enable representation learning without any supervised signal. GANs and VAEs learn representations by directly transforming latent codes to data samples. In contrast, the introduced diffusion-based representation learning relies on a new formulation of the denoising score matching objective and thus encodes the information needed for denoising. We illustrate how this difference allows for manual control of the level of details encoded in the representation. Using the same approach, we propose to learn an infinite-dimensional latent code that achieves improvements of state-of-the-art models on semi-supervised image classification. We also compare the quality of learned representations of diffusion score matching with other methods like autoencoder and contrastively trained systems through their performances on downstream tasks.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: We changed the wording from "Handicapped" to "Restricted" for describing the weaker version of SimCLR, as well as added clarifications to the proposition and more discussion on related works.
Assigned Action Editor: ~Mohammad_Norouzi1
Submission Number: 178
Loading