Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations

Litu Rout, Yujia Chen, Nataniel Ruiz, Constantine Caramanis, Sanjay Shakkottai, Wen-Sheng Chu

Keywords: Inverse Problems, Generative Modeling, Diffusion Models, Rectified Flows, Posterior Sampling, Optimal Control

TL;DR: We present an efficient inversion method for rectified flow models, including Flux, that requires no additional parameter training, latent variable optimization, prompt tuning, or complex attention processors.

Abstract:

Generative models transform random noise into images, while their inversion aims to reconstruct structured noise for recovery and editing.
This paper addresses two key tasks: (i) *inversion* and (ii) *editing* of real images using stochastic equivalents of rectified flow models (e.g., Flux).
While Diffusion Models (DMs) dominate the field of generative modeling for images, their inversion suffers from faithfulness and editability challenges due to nonlinear drift and diffusion.
Existing DM inversion methods require costly training of additional parameters or test-time optimization of latent variables.
Rectified Flows (RFs) offer a promising alternative to DMs, yet their inversion remains underexplored.
We propose RF inversion using dynamic optimal control derived via a linear quadratic regulator, and prove that the resulting vector field is equivalent to a rectified stochastic differential equation.
We further extend our framework to design a stochastic sampler for Flux.
Our method achieves state-of-the-art performance in zero-shot inversion and editing, surpassing prior works in stroke-to-image synthesis and semantic image editing, with large-scale human evaluations confirming user preference.
See our project page https://rf-inversion.github.io/ for code and demo.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1202