Manufacture Assembly Fault Detection Method based on Deep Learning and Mixed Reality

Shuai Wang, Ruifeng Guo, Hongliang Wang, Yuanjing Ma, Zixiao Zong

Abstract: Traditional manufacture assembly fault detection method more relied on manual operation leading to an ineffective procedure. In order to solve the problem making mistakes caused by manual inspection such as missing installations or lack of visual and intuitive guidance in traditional manufacturing assembly, A method of manufacture assembly fault detection based on deep learning and mixed reality is proposed. By training a pretreatment model and detecting targets via Faster R-CNN convolutional neural network, the accurate and efficient assembly detection is realized in the manufacturing by extracting feature information. Meanwhile, the mixed reality combination is realized by using the mixed reality device HoloLens as the human-computer interaction module to access the system. Augmented information interactions make the manufacturing and assembly inspection process more visual and intuitive. The experimental results show that the method proposed has a higher detection accuracy, a better user experience and a great capacity on an efficient assembly detection.