Provably Efficient Scheduling of Dynamically Allocating Programs on Parallel Cache Hierarchies

Guy E. Blelloch, Phillip B. Gibbons, Harsha Vardhan Simhadri

Abstract: Thread schedulers are designed to dynamically map parallel programs to processors to optimize performance metrics including memory footprint, number of cache misses at each cache level, and load balance, so as to minimize the total running time of the program. Programs with dynamic memory allocation pose particular challenges for thread schedulers, and indeed prior schedulers that are provably cache- and time-efficient on multi-level cache hierarchies require static memory allocation. Not only do many thread schedulers fail to reuse memory effectively, but there is often an inherent trade-off between parallelism and memory use in algorithms. In this paper, we present the first runtime thread scheduler for multi-level cache hierarchies, called the space-bounded recursive-PDF scheduler, that is provably space-, cache-, and time-efficient for parallel programs that dynamically allocate memory. Our bounds hold for nested parallel programs with good regularity as measured by the effective cache complexity — a program-centric metric. The cache and time bounds are asymptotically optimal, while the space bound is asymptotically optimal for highly parallel and regular programs.