Digital Twin of a Multi-Arm Robot Platform based on Isaac Sim for Synthetic Data Generation

Juan Jose Quiroz Omana, Murilo Marques Marinho, Kanako Harada

Keywords: Synthetic Data, Digital-Twins, Semantic Segmentation, Convolutional Neural Networks, Isaac Sim

Abstract: Autonomous robotic surgery combines state-of-the-art strategies to potentially provide more efficacy and safety regardless of the surgeon’s skill. These approaches usually use CNNs, which require a large amount of data for suitable training. However, in some applications as medical procedures on animals, performing thousands of trials would be ethically hard to justify. In this letter, we develop a digital twin on Isaac Sim to create a synthetic dataset. We use synthetic images to train a CNN for image segmentation tasks of our AI robot science (AISP) platform. We compare it to a second CNN trained with real images showing that for a specific validation dataset, the CNN trained with synthetic images performs similarly to the one trained with real images.