Towards Overhead Semantic Part Segmentation of Workers in Factory Environments from Depth Images using a FCN

Masakazu Yoshimura, Murilo M. Marinho, Atsushi Nakazawa, Kanako Harada, Mamoru Mitsuishi, Takuya Maekawa, Yasuo Namioka

Abstract: In production lines, human workers assemble and/or inspect products in a predetermined process flow. Training new workers is a complex process with varying results. To track the workers motion in a way to better understand human skill, an overhead semantic part segmentation of workers is desired. For this purpose, in this work, we propose a fully-convolutional neural-network model paired with four proposed augmentation strategies. Artificial depth images were used as training data and the augmentation strategies were essential in aiding the network to generalize to the real images. The proposed method was evaluated in two tasks with different backgrounds: a part assembly task and a quality check task. We improved the F-measure by 12% in the part assembly task and 4% in the quality check task when compared to our previous work.