SESR: Self-Ensembling Sim-to-Real Instance Segmentation for Auto-Store Bin Picking

Biqi Yang, Xiaojie Gao, Kai Chen, Rui Cao, Yidan Feng, Xianzhi Li, Qi Dou, Chi-Wing Fu, Yun-Hui Liu, Pheng-Ann Heng

Abstract: Instance segmentation is an important task for supporting robotic grasping in auto-store scenarios. Accurate segmentation usually relies on the quantity and quality of available annotated training data. However, it requires tremendous cost to obtain these labels. In this work, without requiring any human annotations on real data, our proposed self-ensembling sim-to-real network, namely SESR, is able to generate precise instance masks for a wide variety of supermarket goods. We design our SESR with a teacher model and a student model trained with a self-ensembling strategy. We adopt different levels of consistency to bridge the sim-to-real gap and boost the model generalization ability. Also, we compile an auto-store bin-picking dataset covering various goods. Extensive experiments on both unseen scenarios and unseen objects validate the effectiveness and superiority of our method over others, and the robot arm demonstrations further show that our segmentation results can support real-time auto-store bin picking.