Efficient Automatic Scheduling of Imaging and Vision Pipelines for the GPU

Luke Anderson, Andrew Adams, Karima Ma, Tzu-Mao Li, Tian Jin, Jonathan Ragan-Kelley

Abstract:

We present a new algorithm to quickly generate high-performance GPU implementations of complex imaging
and vision pipelines, directly from high-level Halide algorithm code. It is fully automatic, requiring no schedule
templates or hand-optimized kernels. We address the scalability challenge of extending search-based automatic
scheduling to map large real-world programs to the deep hierarchies of memory and parallelism on GPU
architectures in reasonable compile time. We achieve this using (1) a two-phase search algorithm that first
‘freezes’ decisions for the lowest cost sections of a program, allowing relatively more time to be spent on
the important stages, (2) a hierarchical sampling strategy that groups schedules based on their structural
similarity, then samples representatives to be evaluated, allowing us to explore a large space with few samples,
and (3) memoization of repeated partial schedules, amortizing their cost over all their occurrences. We guide
the process with an efficient cost model combining machine learning, program analysis, and GPU architecture
knowledge.
We evaluate our method’s performance on a diverse suite of real-world imaging and vision pipelines. Our
scalability optimizations lead to average compile time speedups of 49× (up to 530×). We find schedules that
are on average 1.7× faster than existing automatic solutions (up to 5×), and competitive with what the best
human experts were able to achieve in an active effort to beat our automatic results.