TranDRL: A Transformer-Driven Deep Reinforcement Learning Enabled Prescriptive Maintenance Framework

Yang Zhao, Jiaxi Yang, Wenbo Wang, Helin Yang, Dusit Niyato

Abstract: Industrial systems require reliable predictive maintenance strategies to enhance operational efficiency and reduce downtime. Existing studies rely on the heuristic models which may struggle to capture complex temporal dependencies. This article introduces an integrated framework that leverages the capabilities of the Transformer and deep reinforcement learning (DRL) algorithms to optimize the system maintenance actions. Our approach employs the Transformer model to effectively capture complex temporal patterns in IoT sensor data, thus accurately predicting the remaining useful life (RUL) of equipment. Additionally, the DRL component of our framework provides cost-effective and timely maintenance recommendations. Numerous experiments conducted on the NASA C-MPASS data set demonstrate that our approach has a performance similar to the ground truth results and could be obviously better than the baseline methods in terms of RUL prediction accuracy as the time cycle increases. Additionally, the experimental results demonstrate the effectiveness of optimizing maintenance actions.