SchedP: I/O-aware Job Scheduling in Large-Scale Production HPC Systems

Kaiyue Wu, Jianwen Wei, James Lin

Abstract: Job schedulers on High Performance Computing systems can serve more purposes than just maximising computing resource utilisation if they are equipped with more awareness on other aspects of the system. In this work, we focus on making a job scheduler I/O-aware to assist system I/O management. We propose SchedP as the first practical effort on I/O-aware job scheduling that can work in production HPC environment. It trains neural network model to predict each job’s I/O pattern, then makes a delay decision if starting a job right away will lead to I/O congestion in the system. We integrate it into Slurm and performed evaluations with real HPC workloads in production environment for about a month. The results show: a) the neural network model of SchedP reached over 99% for both training and test accuracy on predicting jobs’ I/O patterns; b) SchedP has obvious effect on alleviating system I/O contention.