DynaVect: Context-Aware Modulation of Global Edit Direc- tions for Controllable GAN Editing

TMLR Paper5703 Authors

Abstract: Text-guided editing of generative models like StyleGAN has become a popular method for image manipulation. Current approaches face a trade-off. Optimization-based methods produce edits that are too subtle. This fails to meet the user’s intent for changes. On the flip side, methods that use a single global edit vector often cause unwanted attribute entanglement and identity loss. In this work, we propose DynaVect, a hybrid approach that attempts to resolve this trade-off. Our approach is a lightweight Dynamic Contextual Modulator. The DCM is a network trained to predict a personalized correction (or delta), based on the source image’s features. At inference time, this learned delta is used to steer the global edit direction. This results in changes that are visually different while attempting to preserve the original identity. We train our modulator using an optimization-distillation technique. This technique involves creating a fast feed-forward model that approximates the quality of slow, per-image optimization. We demonstrate that our method produces quali- tatively superior results that better align with users expectations as compared to traditional metrics.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=haiXeQFCJu

Changes Since Last Submission:

1. Removed the unused Adaptive Loss Balancer (ALB) and clarified it’s out of scope.
2. Expanded benchmarks to include SDimg2img and Instructpix2pix.
3. Added metric–preference analysis (Spearman/Kendall): CLIP modestly correlates with 2AFC. LPIPS doesn’t. user study is primary.
4. New experiments & ablations: α sweep, protected sets, GlobalOnly/Global+Ortho baselines. Included plus real-image (e4e+PTI, n=100) and zero-shot cars/churches. Also expanded survey to include non-face images
5. More figures/tables (architecture, qualitative, training curves) and cleaned citations/bib to match the paper.

Assigned Action Editor: ~Wei_Liu3

Submission Number: 5703