Lori: Local Low-Rank Response Imputation for Automatic Configuration of Contextualized Artificial Intelligence

Xiaoyu Chen, Ran Jin

Abstract: Artificial intelligence (AI) has played an important role for data-driven decision making in complex engineering problems. However, there has been a huge waste of efforts to configure AI methods (e.g., to select preprocessing and modeling methods, etc.), catering to different contexts (e.g., data analytics objectives, data distributions, etc.). In current practice, data scientists need to manually configure the AI methods in trial-and-errors according to a specific context, including determining the different options of the pipeline components and evaluating the advantages and limitations of an AI method. In this article, we propose a local low-rank response imputation (Lori) method, which will automatically configure AI methods to specific contexts by completing a sparse context-pipeline response matrix. Different from the traditional recommendation systems, Lori performs multivariate partition of the entire context-pipeline response matrix based on the principal Hessian directions of the low-rank imputed response matrix. Thus, the partitioned local low-rank response matrices can be closely modeled to automatically match the AI methods with the datasets. A small-scale and a large-scale case studies in three manufacturing processes demonstrated the merits of the proposed Lori method.