Deep Reinforcement Learning Based Genetic Framework for Flexible Job-Shop Scheduling under Practical Constraints

Kjell van Straaten, Robbert Reijnen, Zaharah Bukhsh, Yaoxin Wu, Yingqian Zhang

Confirmation: I have read and agree with the workshop's policy on behalf of myself and my co-authors.

Tracks: Main Track

Keywords: Deep reinforcement learning, Flexible job-shop scheduling problem, Genetic algorithm

Abstract: In this paper, we propose a DRL-based genetic framework (DRL-GF) for solving flexible job shop scheduling problems (FJSPs) under various practical constraints in real-world applications. First, we model the genetic algorithm (GA) process as a Markov decision process (MDP). Then, we use a double-layer encoding scheme to represent the population of schedules for an FJSP instance, where we develop a set of problem-agnostic features to describe the state of the GA solution process. We train a multilayer perceptron (MLP) using a proximal policy optimization (PPO) algorithm to determine the mutation probability, crossover probability, and mutation rate simultaneously. We evaluate the proposed DRL-GF on the standard FJSP instances and the FJSP with sequence-dependent setup time (SDST). Moreover, we test our method on real-world FJSP instances with additional practical constraints. Extensive results demonstrate that DRL-GF outperforms conventional heuristics and end-to-end DRL methods in each scenario, requiring minimal problem-specific customization. In addition, we show that even if we train DRL-GF using the classical FJSP instances, the learned policy can be used directly to solve the heavily constrained FJSP-SRMN, greatly outperforming the benchmarked methods.

Submission Number: 30