Image Based Virtual Try-on Network from Unpaired Data

Assaf Neuberger, Eran Borenstein, Bar Hilleli, Eduard Oks, Sharon Alpert

Abstract:

This paper presents a new image-based virtual try-on
approach (Outfit-VITON) that helps visualize how a composition of clothing items selected from various reference
images form a cohesive outfit on a person in a query image.
Our algorithm has two distinctive properties. First, it is inexpensive, as it simply requires a large set of single (noncorresponding) images (both real and catalog) of people
wearing various garments without explicit 3D information.
The training phase requires only single images, eliminating
the need for manually creating image pairs, where one image shows a person wearing a particular garment and the
other shows the same catalog garment alone. Secondly, it
can synthesize images of multiple garments composed into
a single, coherent outfit; and it enables control of the type
of garments rendered in the final outfit. Once trained, our
approach can then synthesize a cohesive outfit from multiple images of clothed human models, while fitting the outfit
to the body shape and pose of the query person. An online
optimization step takes care of fine details such as intricate
textures and logos. Quantitative and qualitative evaluations
on an image dataset containing large shape and style variations demonstrate superior accuracy compared to existing state-of-the-art methods, especially when dealing with
highly detailed garments.