Does the Manipulation Process Matter? RITA: Reasoning Composite Image Manipulations via Reversely-Ordered Incremental-Transition Autoregression

Xuekang Zhu, Ji-Zhe Zhou, Kaiwen Feng, Chenfan Qu, Yunfei Wang, Liting Zhou, Jian liu

Keywords: image manipulation localization, IML

Abstract:

Image manipulations often entail a complex manipulation process, comprising a series of editing operations to create a deceptive image, exhibiting **sequentiality** and **hierarchical** characteristics. However, existing IML methods remain manipulation-process-agnostic, directly producing localization masks in a one-shot prediction paradigm without modeling the underlying editing steps. This one-shot paradigm compresses the high-dimensional compositional space into a single binary mask, inducing severe **dimensional collapse**, thereby creating a fundamental mismatch with the intrinsic nature of the IML task.

To address this, we are the first to reformulate image manipulation localization as a conditional sequence prediction task, proposing the **RITA** framework. RITA predicts manipulated regions layer-by-layer in an ordered manner, using each step's prediction as the condition for the next, thereby explicitly modeling temporal dependencies and hierarchical structures among editing operations.

To enable training and evaluation, we synthesize multi-step manipulation data and construct a new benchmark **HSIM**. We further propose the **HSS** metric to assess sequential order and hierarchical alignment. Extensive experiments show RITA achieves SOTA on traditional benchmarks and provides a solid foundation for the novel hierarchical localization task, validating its potential as a general and effective paradigm. The code and dataset will be publicly available.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 3039