Gen2Det: Generate to Detect

Saksham Suri, Fanyi Xiao, Animesh Sinha, Sean Culatana, Raghuraman Krishnamoorthi, Chenchen Zhu, Abhinav Shrivastava

Keywords: Object Detection and Segmentation, Synthetic Data, Diffusion Models

TL;DR: Pipeline to use state-of-the-art diffusion models as synthetic data sources to improve object detection and segmentation models.

Abstract: Diffusion models have shown improvement in synthetic image quality as well as better control in generation. We motivate and present Gen2Det, a simple modular pipeline to create synthetic training data for object detection by leveraging state-of-the-art grounded generation methods. Unlike existing works which generate individual object instances, require identifying foreground followed by pasting on other images, we simplify to directly generating scene-centric images. In addition to the synthetic data, Gen2Det also proposes a suite of techniques to best utilize the generated data, including image-level filtering, instance-level filtering, and better training recipe to account for imperfections in the generation. Using Gen2Det, we show healthy improvements on object detection and segmentation tasks on standard benchmarks like COCO and LVIS.

Submission Number: 32