Alterbute: Editing Intrinsic Attributes of Objects in Images

Tal Reiss, Daniel Winter, Matan Yaakov Cohen, Alex Rav-Acha, Yael Pritch, Ariel Shamir, Yedid Hoshen

TL;DR: Presenting Alterbute, a new diffusion-based method that edits an object's intrinsic attributes (color, texture, material, shape) in images while preserving its identity.

Abstract: We introduce Alterbute, a diffusion-based method for editing an object's intrinsic attributes in an image. We allow changing color, texture, material, and even the shape of an object, while preserving its perceived identity and scene context. Existing approaches either rely on unsupervised priors that often fail to preserve identity or use overly restrictive supervision that prevents meaningful intrinsic variations. Our method relies on: (i) a relaxed training objective that allows the model to change both intrinsic and extrinsic attributes conditioned on an identity reference image, a textual prompt describing the target intrinsic attributes, and a background image and object mask defining the extrinsic context. At inference, we restrict extrinsic changes by reusing the original background and object mask, thereby ensuring that only the desired intrinsic attributes are altered; (ii) Visual Named Entities (VNEs) - fine-grained visual identity categories (e.g., "Porsche 911 Carrera") that group objects sharing identity-defining features while allowing variation in intrinsic attributes. We use a vision-language model to automatically extract VNE labels and intrinsic attribute descriptions from a large public image dataset, enabling scalable, identity-preserving supervision. Alterbute outperforms existing methods on identity-preserving object intrinsic attribute editing.

Lay Summary:

AI image editing tools can change what objects look like in photos, but they
struggle with a fundamental challenge: when you change one property of an
object, say, making a wooden chair look like marble, the tool often changes
the object itself, producing a generic marble chair instead of preserving the
specific chair's design. Current methods lack the ability to distinguish
between what an object is and what it looks like.

We developed Alterbute, a method that learns this distinction by training on
pairs of images showing the same product in different materials, colors, shapes, or
textures. By seeing, for example, the same sneaker model in leather and in
canvas, the model learns which visual features define the object's identity
and which can be changed.

This enables more precise and reliable image editing in settings where
preserving an object's specific identity during visual changes is essential.

Primary Area: Applications->Computer Vision

Keywords: Image editing, intrinsic attribute editing

Submission Number: 22558