Boosting Instance Segmentation with Synthetic Data: A study to overcome the limits of real world data sets

Florentin Poucin, Andrea Kraus, Martin Simon

Abstract:

A major issue related to computer vision for the automo-
tive industry is that real-world perception models require
huge amount of well-annotated data to achieve decent per-
formance. While this data is very expensive to collect and
annotate, synthetically generated images seem to be an ef-
ficient alternative to solve this problem. More and more
public data sets, composed of synthetic data, are avail-
able in various domains, however, there is too little con-
crete methodology to use them properly. In this paper, we
propose a simple approach combining the use of synthetic
and real images to boost instance segmentation. We men-
tion some pre-processing requirements as harmonizing in-
stance labeling and removing non-valuable instances from
synthetic images. We present our training strategy based on
data set mixing, and show that it overcomes the domain shift
between real and synthetic data sets. A comparison study
with other training approaches, such as fine-tuning tech-
niques, highlights the benefits of our method, which boosts
network performances on both real and synthetic image in-
ferences.