A Scheduling optimization Mechanism Combining Q-learning and Genetic Algorithm

Xue Wang, Xingwei Wang, Jie Jia, Xijia Lu, Min Huang

Abstract: In recent years, the number of network applications is constantly increasing, and network congestion often occurs. To ensure the network Quality of Service (QoS), different types of traffic are classified according to their requirements, and similar traffic is transmitted to the same queue for scheduling. The switch generally uses fair queuing and its extension schemes to schedule traffic. These schemes achieve different bandwidth allocation by configuring different queue weights, so as to obtain a lower packet loss rate. However, the switch can provide us with very few statistical parameters, so using a large number of statistical parameters for adaptive weight adjustment is challenging in implementation. At the same time, the weight range supported by the switch is large, but the action space supported by reinforcement learning is limited, which cannot represent the entire queue weight space. Although deep reinforcement learning can solve the problem with large space, the existing switches can not well support the calculation of neural network model. In this paper, we propose a scheduling optimization mechanism combining Q-learning and genetic algorithm, called QGSO, which is used to schedule traffic in real switches. Firstly, we model the scheduling optimization problem as a Markov decision process (MDP) and use Q-learning to solve it in order to select the optimal queue weights according to the state of the environment. Secondly, we use genetic algorithm to filter out a group of optimal queue weights from the weight space, achieving compression of the solution space. Finally, we use a hardware testbed to test and verify the effectiveness of the algorithm. The experimental results show that our algorithm can effectively schedule traffic and achieve a lower packet loss rate.