Preserving Product Fidelity in Large Scale Image Recontextualization with Diffusion Models

Ishaan Malhi, Praneet Dutta, Ellie Talius, Sally Ma, Brendan Driscoll, Krista Holden, Garima Pruthi, Arunachalam Narayanaswamy

Keywords: Diffusion Models, Product Recontextualization, Synthetic Data Augmentation, Novel View Synthesis, E-commerce

TL;DR: We present a framework for high-fidelity product image recontextualization using diffusion models and a novel synthetic data augmentation pipeline, enabling realistic and diverse product visualizations for applications like e-commerce.

Abstract: We present a framework for high-fidelity product image recontextualization using text-to-image diffusion models and a novel data augmentation pipeline. This pipeline leverages image-to-video diffusion, in/outpainting, and counterfactual generation to create synthetic training data, addressing limitations of real-world data collection for this task. Our method improves the quality and diversity of generated images by disentangling product representations and enhancing the model's understanding of product characteristics. Evaluation on the ABO dataset and a private product dataset, using automated metrics and human assessment, demonstrates the effectiveness of our framework in generating realistic and compelling product visualizations, with implications for diverse applications such as e-commerce and virtual product showcasing.

Submission Number: 69