TMoE-P: Toward the Pareto Optimum for Multivariate Soft Sensors

Licheng Pan, Hao Wang, Zhichao Chen, Yuxin Huang, Zhaoran Liu, Qunshan He, Xinggao Liu

Abstract: Multivariate soft sensors seek to provide accurate estimation of multiple quality variables through the analysis of measurable process variables, representing a significant advance over the traditional focus on single-quality variable sensors within industrial manufacturing. Current progress stays in applying parameter-sharing neural architectures while ignoring two fundamental issues: (1) catastrophic interference, where the indiscriminate sharing of parameters degrades performance due to the discrepancy of objectives; (2) seesaw optimization, where the optimizer overly focuses on one dominant yet simple objective at the expense of others. To address these issues, we reformulate multivariate soft sensors as a multi-objective optimization problem and propose the Task-aware Mixture-of-Experts framework for achieving the Pareto optimum (TMoE-P). Specifically, to handle issue (1), we propose an Objective-aware Mixture-of-Experts (OMoE) module, which consists of objective-specific and objective-shared experts to realize parameter sharing while accommodating the discrepancy between objectives. To handle issue (2), we devise a Pareto Objective Weighting (POW) module, which dynamically balances the weights of learning objectives to approximate the Pareto optimum among competing objectives. Our evaluations on a public soft sensor benchmark showcase TMoE-P’s superior performance, confirming its enhanced accuracy and robustness. Note to Practitioners—Addressing the burgeoning complexity of estimating multiple quality variables in industrial manufacturing processes, this study introduces a novel Task-aware Mixture-of-Experts framework aiming for the Pareto Optimum (TMoE-P). This framework mitigates the issues of catastrophic interference and seesaw optimization by achieving a delicate balance between parameter sharing and maintaining distinctness between objectives, guiding towards the Pareto optimum. The empirical findings affirm the framework’s capability to adeptly handle the complex interplay of variables, potentially enhancing the efficiency and reliability of industrial processes. While initially developed for multivariate data in industrial applications, the TMoE-P framework’s modular and adaptable nature facilitates its extension to a broad spectrum of data structures, optimizers, and neural architectures. Its adaptability offers practitioners a valuable tool to fulfill specific task requirements, making a modest contribution to industrial process control and monitoring.