Processing Product, Production and Producer Information for Operations Planning and Scheduling Using CLIP for Multimodal Image and Text Data

Julia Markert, Matthias Kerzel, Michael Variola, Dominik Saubke, Stephanie von Riegen, Emad Aghajanzadeh, Lothar Hotz, Pascal Krenz

Abstract: Recently, interest in producing more locally has risen due to, e.g., the climate crisis and supply chain issues. This increasing demand for local production creates new opportunities, but often also challenges for micro and small local enterprises. Collaborating in production networks as a means to join forces and resources can therefore be of great advantage to them. Operations Planning and Scheduling in such a network across companies is a difficult task, that could benefit from the use of information processing and Artificial Intelligence. One promising technology for this application is CLIP, which was introduced in 2021 by Open AI. It is a neural network that uses text-image pairs, and the acronym stands for “Contrastive Language-Image Pre-training”. This paper is an expansion on previous work to explore and test ways in which CLIP can be utilized to support Operations Planning and Scheduling (OPS), especially in local production networks, using real-world data in the form of text and images. It is shown in this paper that combining these modalities can enhance downstream tasks like classification or similarity analysis.