Metalearning Using Structure-rich Pipeline Representations for Better AutoML

Anonymous

Keywords: metalearning, automl, machine learning pipeline

TL;DR: AutoML systems that use metalearning can rank and estimate the performance of machine learning pipelines better when using the complete pipeline structure metadata.

Abstract: Automatic machine learning (AutoML) systems have been shown to perform better when they learn from past experience. Examples include Auto-sklearn, which warm-starts the ML pipeline search using existing programs known to perform well on "similar" tasks, and AlphaD3M, which uses online reinforcement learning to search the ML pipeline space. These metalearning approaches, as well as many others, depend on simplifying assumptions about the pipeline search space and/or the pipeline representation. We show that we can estimate ML pipeline performance without simplifying the representation of the pipeline structure. We evaluate this approach on tabular classification tasks and find that it produces the best estimates of pipeline performance and yields pipeline rankings with the highest normalized discounted cumulative gain (nDCG) and the lowest regret.

Proposed Reviewers:

Brandon Schoenfeld,[e-mail]
Orion Weller,[e-mail]