The Unanticipated Asymmetry Between Perceptual Optimization and Assessment

Jiabei Zhang, WANG QI, Siyu Wu, Du Chen, Tianhe Wu

Keywords: Perceptual Optimization, Image Quality Assessment, Fidelity Metrics, Adversarial Objective

TL;DR: This work systematically examines the unanticipated asymmetry between perceptual optimization and assessment.

Abstract: Perceptual optimization is primarily driven by the fidelity objective, which enforces both semantic consistency and overall visual realism, while the adversarial objective provides complementary refinement by enhancing perceptual sharpness and fine-grained detail. Despite their central role, the correlation between their effectiveness as optimization objectives and their capability as image quality assessment (IQA) metrics remains underexplored. In this work, we conduct a systematic analysis and reveal an unanticipated asymmetry between perceptual optimization and assessment: fidelity metrics that excel in IQA are not necessarily effective for perceptual optimization, with this misalignment emerging more distinctly under adversarial training. In addition, while discriminators effectively suppress artifacts during optimization, their learned representations offer only limited benefits when reused as backbone initializations for IQA models. Beyond this asymmetry, our findings further demonstrate that discriminator design plays a decisive role in shaping optimization, with patch-level and convolutional architectures providing more faithful detail reconstruction than vanilla or Transformer-based alternatives. These insights advance the understanding of loss function design and its connection to IQA transferability, paving the way for more principled approaches to perceptual optimization. Code and models will be released publicly.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 3077