MDQ: A QoS-Congestion Aware Deep Reinforcement Learning Approach for Multi-Path Routing in SDN

Lizeth Patricia Aguirre Sanchez, Yao Shen, Minyi Guo

Abstract: The challenge of link overutilization in networking persists, prompting the development of load-balancing methods such as multi-path strategies and flow rerouting. However, traditional rule-based heuristics struggle to adapt dynamically to network changes. This leads to complex models and lengthy convergence times, unsuitable for diverse QoS demands, particularly in time-sensitive applications. Existing routing approaches often result in specific types of traffic overloading links or general congestion, prolonged convergence delays, and scalability challenges. To tackle these issues, we propose a QoS-Congestion Aware Deep Reinforcement Learning Approach for Multi-Path Routing in Software-Defined Networking (MDQ). Leveraging Deep Reinforcement Learning, MDQ intelligently selects optimal multi-paths and allocates traffic based on flow needs. We design a multi-objective function using a combination of link and queue metrics to establish an efficient routing policy. Moreover, we integrate a congestion severity index into the learning process and incorporate a traffic classification phase to handle mice-elephant flows, ensuring that diverse class-of-service requirements are adequately addressed. Through an RYU-Docker-based Openflow framework integrating a Live QoS Monitor, DNC Classifier, and Online Routing, results demonstrate a 19%–22% reduction in delay compared to state-of-the-art algorithms, exhibiting robust reliability across diverse scenarios of network dynamics.