Taming Rectified Flow for Inversion and Editing

Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, Ying Shan

Abstract: Rectified-flow-based diffusion transformers like FLUX and OpenSora have demonstrated outstanding performance in the field of image and video generation. Despite their robust generative capabilities, these models often struggle with inversion inaccuracies, which could further limit their effectiveness in downstream tasks such as image and video editing. To address this issue, we propose RF-Solver, a novel training-free sampler that effectively enhances inversion precision by mitigating the errors in the ODE-solving process of rectified flow. Specifically, we derive the exact formulation of the rectified flow ODE and apply the high-order Taylor expansion to estimate its nonlinear components, significantly enhancing the precision of ODE solutions at each timestep. Building upon RF-Solver, we further propose RF-Edit, a general feature-sharing-based framework for image and video editing. By incorporating self-attention features from the inversion process into the editing process, RF-Edit effectively preserves the structural information of the source image or video while achieving high-quality editing results. Our approach is compatible with any pre-trained rectified-flow-based models for image and video tasks, requiring no additional training or optimization. Extensive experiments across generation, inversion, and editing tasks in both image and video modalities demonstrate the superiority and versatility of our method. The source code is available at https://github.com/wangjiangshan0725/RF-Solver-Edit.

Lay Summary:

Many rectified-flow diffusion transformers can generate realistic images and videos but struggle to convert inputs back into precise latent codes, which limits editing accuracy.
RF-Solver tackles this by deriving the exact rectified flow ODE and using high-order Taylor expansions to reduce ODE-solving errors at each timestep without any extra training. RF-Edit then shares self-attention features from the inversion stage into the editing stage to preserve structural details during edits.
Extensive tests across generation, inversion, and editing tasks show more faithful inversions and higher-quality edits with no retraining required.

Link To Code: https://github.com/wangjiangshan0725/RF-Solver-Edit

Primary Area: Applications->Computer Vision

Keywords: Diffusion; Image Editing; Video Editing

Submission Number: 9468