Towards machinery incremental fault diagnosis based on inverted transformer lifelong learning with learnable pruning mechanism

Haihong Tang, Jia-wei Li, Wu-wei Feng, Peng Chen, Hong-tao Xue

Abstract: To overcome catastrophic forgetting in deep learning for bearing diagnosis in wind turbines, it is necessary to boost stability-plasticity in lifelong learning that ensures the generation of high-quality exemplars translated from time-series signals in numerous sensors while incrementally learning multiple fresh classes. Therefore, an Inverted transformer lifetime learning method is forwarded to address the abovementioned limitations without tedious retraining for machinery fault diagnosis. First, the backbone of this method is the Inverted Transformer, which independently embeds the time-series signals of every sensor into tokens that simultaneously aggregate the global representations of series and enlarge the local receptive field via booming attention mechanisms. Second, the Inverted transformer expansion is developed to enable learning new and old knowledge by adding new branches based on the Inverted transformer to incrementally learn multiple new classes. Next, the learnable pruning mechanism is introduced to alleviate the dilemma caused by predefined and fixed structures in the previous stage and enhance the learning ability of the added fresh branch. Finally, a multi-objective training strategy is designed to overcome the class imbalance issues induced by several faults added in the incremental stage. The experimental results demonstrate the effectiveness and feasibility of the novel lifelong learning method.