ScoreFlow: Bridging Score and Neural ODE for Reversible Generative Modeling

Dong Pang, Xinyi Le

Keywords: Reversible generative modeling, Neural ODEs, Diffusion models, Image translation, Deep learning

Abstract: Neural ordinary differential equations (ODEs) are commonly used in reversible generative models. However, training neural ODEs is computationally expensive for estimating the log-likelihood density and backpropagating through ODE solvers, leading to slow convergence and significant gradient estimation errors. This paper presents ScoreFlow, a novel generative model capable of reversible and controllable data transformations. Firstly, we formulate an ODE utilizing a score variant as the drift term to model transformations between two certain data distributions. Secondly, we suggest a path-constrained loss to reduce truncation errors, enhancing the model's capabilities in generating high-quality samples. Thirdly, ScoreFlow has the ability to employ a single model to achieve both conditional image generation and cross-class image translation tasks. The closed-form optimal solution for data transformation in ScoreFlow is theoretically proven, providing support for the model's efficient training. Furthermore, the effectiveness of our approach is empirically validated through image generation, translation, and interpolation experiments.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8904