PPS: Fair and efficient black-box scheduling for multi-tenant GPU clusters

Kaihao Ma, Zhenkun Cai, Xiao Yan, Yang Zhang, Zhi Liu, Yihui Feng, Chao Li, Wei Lin, James Cheng

Abstract: Highlights•Existing GPU cluster scheduling strategies trade-off between fairness and efficiency.•Aggregate job statistics are predictable for large-scale production GPU clusters.•Our scheduler PPS achieves both fairness and efficiency by predicting resource status.•PPS is general, transparent, and secure because it is black-box and non-preemptive.•Tailored procedures are designed to predict the idle resources and resource demands.