Contextually Harmonious Local Video Editing
Keywords: Video Editing, Diffusion Models, Contextaul Harmonious
Abstract: We introduce a new task for video editing: Contextual Harmonious Local Editing, which focuses on replacing a local moving subject in videos containing multiple subjects or reference objects. The goal is to ensure that the replaced subject maintains its original motion while its size remains harmonious with the scene's context. Previous methods often face two specific challenges when addressing this task: (1) ensuring the size of the replaced subject remains contextually harmonious (2) maintaining the original motion and achieving subject replacement without being affected by the motion of other subjects. To address the above problems, we propose a novel three stage video editing pipeline. We initially leverage large pre-trained models to acquire knowledge about the shape and size differences between the original and replaced subjects. To mitigate interference from context motion, we erase other moving subjects to extract the target subject's motion and dynamically choose the editing method to preserve the original subject's motion under different shape transformation.
Following that,we seamlessly replace the original subject in the video with the resized edited subject, ensuring its size harmonizes with the video's context.
As the first work to focus on this task, we also provide a high-quality evaluation dataset and metrics to assess the performance of existing methods on this task. Experimental results based on this dataset demonstrate that our method achieves state-of-the-art (SOTA) performance.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8545
Loading