Repositioning the Subject within Image
Abstract: Current image manipulation primarily centers on static manipulation, such as replacing specific regions within an image or altering its overall style. In this paper, we introduce an innovative dynamic manipulation task, subject repositioning. This task involves relocating a user-specified subject to a desired position while preserving the image's fidelity. Our research reveals that the fundamental sub-tasks of subject repositioning, which include filling the void left by the repositioned subject, reconstructing obscured portions of the subject and blending the subject to be consistent with surrounding areas, can be effectively reformulated as a unified, prompt-guided inpainting task. Consequently, we can employ a single diffusion generative model to address these sub-tasks using various task prompts learned through our proposed task inversion technique. Additionally, we integrate pre-processing and post-processing techniques to further enhance the quality of subject repositioning. These elements together form our SEgment-gEnerate-and-bLEnd (SEELE) framework. To assess SEELE's effectiveness in subject repositioning, we assemble a real-world subject repositioning dataset called ReS. Results of SEELE on ReS demonstrate its efficacy. Code and ReS dataset are available at https://yikai-wang.github.io/seele/.
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=f39wjTjpiB&nesting=2&sort=date-desc
Changes Since Last Submission: We revised our paper based on the feedback from the Action Editor and Reviewers. This includes adding discussions on task inversion, discussion on drag-based image manipulation methods in the related work section, conducting additional experiments and discussion on LoRA methods for local harmonization, subject removal, and completion. We have also made the code and dataset from the paper publicly available.
Code: https://github.com/Yikai-Wang/SEELE-ReS
Assigned Action Editor: ~Zhe_Gan1
Submission Number: 3004
Loading