A Hyper-Heuristic Algorithm with Q-Learning for Distributed Flow Shop-Vehicle Transport-U-Assembly Integrated Scheduling Problem

Dong-Lin Yang, Bin Qian, Zi-Qi Zhang, Rong Hu, Kun Li

Abstract: Distributed Flow Shop-Vehicle Transportation-U-Assembly (DPFS_VT_UA) Integrated Scheduling Problem is a challenging combinatorial optimization problem with many practical applications. In this paper, we propose a Hyper-Heuristic Algorithm with Q-Learning (HHQL) to solve the DPFS_VT_UA integrated scheduling problem. The method combines the advantages of q -learning and hyper-heuristic frameworks. Firstly, according to the characteristics of DPFS_VT_UA, a mixed-integer linear programming (MILP) model of DPFS_VT_UA is established, and an encoding and decoding scheme is designed; secondly, in order to achieve a more in-depth exploration of the solution space for the DPFS_VT_UA problem, the 9 low-level heuristic operators (i.e., 9 effective neighborhood operators) are designed. These low-level heuristics can effectively explore the search space and improve the quality of the solution. The Q-learning mechanism is applied as a high-level strategy to manipulate the Low-Level Heuristics (LLHs), which are then executed in order to search the solution space. Experimental results and statistical analysis show that HHQL significantly outperforms the existing algorithms by a significant margin, demonstrating the effectiveness and efficiency of HHQL in solving DFS_VT_UA integrated scheduling problem.