Scheduling Cluster Tools for Concurrent Processing: Deep Reinforcement Learning With Adaptive Search

Hyun-Jung Kim, Jun-Ho Lee

Abstract: We address the scheduling problem of single-armed cluster tools that concurrently process two wafer types without assuming cyclic scheduling. These cluster tools, consisting of multiple processing modules and a transport robot, are commonly used in semiconductor manufacturing processes, such as etching, deposition, and lithography. To optimize the tool’s throughput, we propose a reinforcement learning approach for determining both the robot task sequence and the release sequence of wafer types. By incorporating an adaptive search, our method intelligently explores future states to gather crucial information, enabling the selection of the best action. Extensive experiments demonstrate that our proposed method outperforms the well-known optimal robot task sequence for cyclic scheduling in single-armed cluster tools with concurrent processing. These findings underscore the effectiveness and superiority of our approach in optimizing the throughput of single-armed cluster tools, without relying on cyclic scheduling assumptions. Note to Practitioners—Single-armed cluster tools, comprising multiple processing modules and a transport robot, are commonly employed in semiconductor manufacturing processes. In recent times, there has been a trend towards concurrent processing of multiple wafer types within the fab, aimed at improving the PMs’ utilization. The concurrent backward sequence (CBS) has emerged as an efficient approach for such scenarios, assuming cyclic scheduling. However, the CBS relies on a cycle plan that specifies the number of wafers from each wafer type to be produced and their release sequence in a cycle. While the CBS provides an optimal schedule that maximizes the throughput of the tool in some cases under the cyclic scheduling assumption, its performance has not been guaranteed for other general cases. We propose a highly efficient reinforcement learning approach for the scheduling problem including the robot task sequence and release sequence of wafer types in a single-armed cluster tool. Our method outperforms the CBS, demonstrating its superior performance. By implementing our proposed approach, we believe that practitioners can effectively maximize the throughput of single-armed cluster tools and significantly enhance the fab productivity.