Variational Inference Over Graph: Knowledge Representation for Deep Process Data Analytics

Zhichao Chen, Zhihuan Song

Abstract: With the advent of the industrial Big Data era, accurate estimation of product quality and monitoring of workingconditions from historical data have become crucial in the pro.cess industry, However, the majority of data-driven approachespredominantly rely on observational data, overlooking the valu-able empirical knowledge derived from experience or underlyingmechanisms. In order to leverage this knowledge, researchers em-ploy various graph neural network-based methods which intro.duce connections among process variables for feature extraction.Nevertheless, it is imperative to recognize that process knowledgeundergoes changes due to internal or external concept drift. Toaddress this challenge, we propose a novel deep learning modulecalled “variational inference over graph,, to effectively harnessshifting knowledge. Building upon the self-attention mechanism,we design a probabilistic self-attention mechanism for encodingand reconciling prior knowledge. Instead of directly encoding theprior knowledge through graph neural network edges, we incor-porate it as regularization term within the variational inferenceframework that accounts for knowledge shift. Furthermore, weintroduce reparameterization estimator to control the varianceresulting from knowledge uncertainty. To showcase the capabilityofour proposed method, we conduct various experiments on qualityprediction task in real industrial processes.