A Working time Deadline-based Backfilling Scheduling solution

Thanh Hoang Le Hai, Kien Trung Pham, Nam Thoai

Abstract: High-Performance Computing systems are playing an essential role in resolving complex real-world problems and attract a significant amount of computing demand. This leads to an urgent need for the optimization of resource usage. Most systems use a Resource and Job Management System (RJMS) that tries to finish all queued jobs as soon as possible. However, completing some jobs quickly may not deliver more benefits to users if they cannot access their outputs immediately. Thus, we can delay these jobs to reserve resources for other jobs which are supposed to be used instantly when they finished. In this paper, we analyze the user behavior on four real workload traces and their effects on job execution patterns. From these insights, we propose a practical deadline determination based on constraints about working time and present our Working time Deadline-based Backfilling (WDBF) algorithm. Any jobs that are estimated to be finished outside the work schedule will be delayed, as long as their outputs are ready before the next working period. The simulation results demonstrate that our practical deadline determination has smaller the average waiting time and stretch value than the random deadline selection on suitable workloads. Additionally, we also show that the WDBF algorithm still reserves the maximum waiting time as compared to the EASY Backfilling and the Conservative Backfilling while performing significantly better than the original Deadline-based backfilling and the Earliest Deadline First algorithm. Moreover, the WDBF also has a higher number of jobs that are submitted and finished in working time than other algorithms in most cases.