Enhancing Consistency of Flow-Based Image Editing through Kalman Control

Haozhe Chi, Zhicheng Sun, Yang Jin, Yi Ma, Jing Wang, Yadong MU

Keywords: Image Generation, Rectified Flows, Generative Modeling, Optimal Control, Image Editing

Abstract: Flow-based generative models have gained popularity for image generation and editing. For instruction-based image editing, it is critical to ensure that modifications are confined to the targeted regions. Yet existing methods often fail to maintain consistency in non-targeted regions between the original / edited images. Our primary contribution is to identify the cause of this limitation as the error accumulation across individual editing steps and to address it by incorporating the historical editing trajectory. Specifically, we formulate image editing as a control problem and leverage the Kalman filter to integrate the historical editing trajectory. Our proposed algorithm, dubbed Kalman-Edit, reuses early-stage details from the historical trajectory to enhance the structural consistency of the editing results. To speed up editing, we introduce a shortcut technique based on approximate vector field velocity estimation. Extensive experiments on several datasets demonstrate its superior performance compared to previous state-of-the-art methods.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 14607