HALO: Human-Aligned End-to-end Image Retargeting with Layered Transformations

Yiran Xu, Harris Shadmany, Siqi Xie, Yinxiao Li, Luciano Sbaiz, Miaosen Wang, Junjie Ke, Jose Lezama, Hang Qi, Han Zhang, Jesse Berent, Ming-Hsuan Yang, Irfan Essa, Jia-Bin Huang, Feng Yang

Keywords: Image Retargeting, Image Transformation, Image Editing

TL;DR: The paper proposes a novel approach that achieves content- and structure-aware image retargeting

Abstract:

Image retargeting aims to change the aspect-ratio of an image while maintaining its content and structure with less visual artifacts.
Existing methods still generate many artifacts or lose a lot of original content or structure. To address this, we introduce HALO, an end-to-end trainable solution for image retargeting.
The core idea of HALO is to warp the input image to target resolution.
Since humans are more sensitive to distortions in salient areas than non-salient areas of an image, HALO decomposes the input image into salient/non-salient layers and applies different wrapping fields to different layers. To further minimize the structure distortion in the output images, we propose perceptual structure similarity loss which measures the structure similarity between input and output images and aligns with human perception. Both quantitative results and a user study on the RetargetMe dataset show that our algorithm achieves SOTA.
Especially, our method increases human preference by 13.21% compared with the second best method.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2629