Cascaded Algorithm Selection With Extreme-Region UCB Bandit

Yi-Qi Hu, Xu-Hui Liu, Shu-Qiao Li, Yang Yu

Abstract: AutoML aims at best configuring learning systems automatically. It contains core subtasks of algorithm selection and hyper-parameter tuning. Previous approaches considered searching in the joint hyper-parameter space of all algorithms, which forms a huge but redundant space and causes an inefficient search. We tackle this issue in a <i>cascaded algorithm selection</i> way, which contains an upper-level process of algorithm selection and a lower-level process of hyper-parameter tuning for algorithms. While the lower-level process employs an <i>anytime</i> tuning approach, the upper-level process is naturally formulated as a multi-armed bandit, deciding which algorithm should be allocated one more piece of time for the lower-level tuning. To achieve the goal of finding the best configuration, we propose the <i>Extreme-Region Upper Confidence Bound</i> (ER-UCB) strategy. Unlike UCB bandits that maximize the mean of feedback distribution, ER-UCB maximizes the extreme-region of feedback distribution. We first consider stationary distributions and propose the ER-UCB-S algorithm that has <inline-formula><tex-math notation="LaTeX">$O(K\ln n)$</tex-math></inline-formula> regret upper bound with <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> arms and <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> trials. We then extend to non-stationary settings and propose the ER-UCB-N algorithm that has <inline-formula><tex-math notation="LaTeX">$O(Kn^\nu)$</tex-math></inline-formula> regret upper bound, where <inline-formula><tex-math notation="LaTeX">$\frac{2}{3}&lt;\nu &lt;1$</tex-math></inline-formula> . Finally, empirical studies on synthetic and AutoML tasks verify the effectiveness of ER-UCB-S/N by their outperformance in corresponding settings.