Partclip: How Does Clip Assist Mechanical Part Image Retrieval?

Shangbo Mao, Dongyun Lin, Aiyuan Guo, Yiqun Li

Abstract: CLIP demonstrates impressive performance across several downstream tasks, such as zero-shot image classification. However, these tasks typically involve images from everyday scenarios, and the efficacy of CLIP in domain-specific com-puter vision tasks associated with the manufacturing industry remains unexplored. This paper first investigates how well CLIP understands the mechanical part images from the man-ufacturing industrial scenes by conducting a thorough eval-uation of its performance in the mechanical part image re-trieval task. It turns out that direct employment of CLIP is less effective for this task. At the same time, considering the requirement of this task for deployment on the industry plat-form in a factory, the large size of the CLIP model presents a practical challenge. Therefore, we explore the knowledge dis-tillation techniques to transfer the knowledge of CLIP into a lighter Efficientnet B1. Our experimental results demonstrate that this CLIP-based knowledge distillation approach can en-hance the performance of Efficientnet B1 on mechanical part image retrieval significantly.