Reliable Routing and Scheduling in Time Sensitive Networks Based on Reinforcement Learning

Hao Cheng, Lei Yang, Qingfeng Zhang, Weiping Zhu

Abstract: Time Sensitive Network (TSN) provides strict low latency and bounded jitter requirements for applications such as industrial systems, autonomous driving, etc. One of the important problems in TSN is to achieve high reliability and low latency by effectively routing and scheduling time-sensitive data flows. Existing work applies heuristic or integer programming to address flow routing and scheduling, yet often fail to achieve optimal solutions quickly. In this paper, we propose a new Reinforcement Learning (RL) based approach for routing and scheduling of redundant data flows, aiming to achieve load balancing on the network links as well as meeting the reliability and delay constraints. Our approach first leverages a simple heuristic algorithm to decide the redundant path candidate set, and then incorporates Proximal Policy Optimization (PPO) method to choose the most suitable multi-routing flows from the candidates, which can be aware of the network status dynamically to reduce the load on the bottleneck link of the network. On this basis, we further retrain the RL model by fine-tuning to adapt to the online environment. The simulation results show that our proposed solution outperforms the benchmark algorithms in terms of the degree of network balance by 38.7% in offline network environments and in terms of average delay by 14.0% in online network environments.