Using Intermediate Forward Iterates for Intermediate Generator Optimization
Abstract: Score-based models have become increasingly popular for image and video generation. In score-based models, a generative task is formulated using a parametric model (such as a neural network) to directly learn the gradient of such high dimensional distributions, instead of the density functions themselves, as is done traditionally. From a mathematical point of view, such gradient information can be utilized in reverse by stochastic sampling to generate diverse samples. However, from a computational perspective, existing score-based models can be efficiently trained only if the forward or the corruption process can be computed in closed form. By using the relationship between the process and layers in a feed-forward network, we derive a backpropagation-based procedure, which we call Intermediate Generator Optimization, to utilize intermediate iterates of the non-Gaussian process with negligible computational overhead. The main advantage of IGO is that it can be incorporated into any standard autoencoder pipeline for generative tasks. We analyze the sample complexity properties of IGO to solve downstream tasks like Generative PCA. We show applications of IGO on two dense predictive tasks, viz., image extrapolation, and point cloud denoising. Our experiments indicate that it is possible to obtain an ensemble of generators for various time points is possible using first-order methods.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Francisco_J._R._Ruiz1
Submission Number: 2878
Loading