Learning-based Image Analytics in User-AI Agent Interactions for Cyber-enabled Manufacturing

Mauro Lemus Alarcon, Upasana Roy, Anirudh Kambhampati, Nguyen P. Nguyen, Minasadat Attari, Ramakrishna Surya, Filiz Bunyak, Matthew R. Maschmann, Kannappan Palaniappan, Prasad Calyam

Abstract: Manual trial-and-error methods are employed for image parameter selection decisions in the processes that control cyber-enabled scientific instruments. Particularly in materials manufacturing use cases, where image analytics can be iterative, time-consuming and prone to errors, there is a need to enhance existing processes by using agents featuring learning algorithms that recommend image analytics parameters for users. In this paper, we present an analysis focused on identifying the optimal learning algorithm for an AI agent guiding real-time image processing tasks in carbon nanotube (CNT) manufacturing, which involves enhancing instrument control of cyber-enabled scanning electron microscope (SEM) instrument image scanning setup parameters such as Zoom, Focus, and Contrast. Specifically, we demonstrate the use of Reinforcement Learning (RL) and Imitation Learning (IL) based agents within a Remote Instrumentation Science Environment (RISE), a modularized system capable of utilizing multiple learning algorithms for guiding image analytics tasks. Further, we conduct a comparative performance analysis of RL and IL using 236CNT images captured in SEM material synthesis experiments. The experiments include configuring scanning parameters for CNT images generated by SEM, implementing CNT image segmentation, and assessing the effectiveness of RL and IL agents in identifying scanning parameters to enhance image quality. The objective is to predict Zoom, Focus, and Contrast parameters using limited labeled data for offline training, guiding dynamic adjustments in SEM settings. Our findings reveal that the IL agent outperforms the RL agent under dynamic conditions in characterizing CNT image parameters - specifically, Zoom, Focus, and Contrast - by evaluating image segmentation metrics.