WinSyn: A High Resolution Testbed for Synthetic Data

Tom Kelly, John Femiani, Peter Wonka

Keywords: Synthetic data, machine learning, rendering, graphics, dataset

TL;DR: Windows. Real Windows. Synthetic Windows. Many Windows.

Abstract:

We present WinSyn, a dataset of high resolution photographs and renderings of 3D models as a testbed for synthetic-to-real research. The dataset consists of 75,739 photographs of building windows, including traditional and modern designs, captured globally. Within these, we supply 89,318 crops containing windows, of which 9,002 are semantically labeled. Further, we present our domain-matched photorealistic procedural model which enables experimentation over a variety of parameter distributions and engineering approaches. Our procedural model provides a second corresponding dataset of 21,000 synthetic images. This jointly developed dataset is designed to facilitate
research in the field of synthetic-to-real learning and synthetic data generation. WinSyn allows experimentation into the factors which make it challenging for synthetic data to complete with real world data. We perform ablations using our synthetic model to identify the salient rendering, materials, and geometric factors pertinent to accuracy within a labeling task. In addition, we leverage our dataset to explore the impact of semi-supervised approaches to synthetic modeling research.

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 987