A Study on the Impact of Domain Randomization for Monocular Deep 6DoF Pose Estimation

Kelvin B. da Cunha, Caio Brito, Lucas Valença, Francisco Simões, Veronica Teichrieb

Abstract: In this work, we apply domain randomization to synthetic images and train deep 6DoF monocular RGB pose estimation models to work on a real object. We compare 19 models trained with different combinations of synthetic and real data (fully synthetic, fully real, initially synthetic and supplemented with real, and a real-synthetic randomized mix). By gradually decreasing the amount of real data used, we show it is possible for deep 6DoF detection to obtain superior results while using less real data (which is harder to obtain) and suggest the best approach to train a model with synthetic data. Our method is validated using a textureless 3D printed object, as the textureless category is a challenging, common open problem in itself. A real and a synthetic dataset generated for this work, totalling over 24,800 annotated frames, are also made public. We also show that synthetic, randomized data can help generalize a model by training it to handle challenges such as illumination changes and fast motion. Finally, we also evaluate how a model trained for one camera sensor works with a different one, and show that synthetic simulations of real cameras can help overcoming this challenge.