Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video

Boxiang Rong, Artur Grigorev, Wenbo Wang, Michael J. Black, Bernhard Thomaszewski, Christina Tsalicoglou, Otmar Hilliges

Keywords: Gaussian Splatting, Garment Reconstruction, Cloth Simulation, Multi-view videos

TL;DR: We reconstruct geometry, appearance and behavior of real garments from multi-view videos.

Abstract:

We introduce Gaussian Garments, a novel approach for reconstructing realistic-looking, simulation-ready, garments from multi-view videos.
Our method represents garments with a combination of a 3D mesh and a Gaussian texture that encodes both the color and high-frequency surface texture.
With this, it is able to model rich textures and surface detail including complex materials such as fur.
As part of our reconstruction process, we demonstrate how such a representation can be used to register a 3D mesh to multi-view videos.
Our reconstructed Gaussian Garments can be animated on unseen body shapes and motions using physics-based simulation.
We show how a learned GNN-based physical simulator can be used to optimize the garment's material parameters to match its real behavior.
Finally, we devise a procedure for automatically ordering virtual garments, allowing us to combine the captured individual garments into multi-garment outfits.

Submission Number: 211