Ada-Boundary: Accelerating the DNN Training via Adaptive Boundary Batch Selection

Hwanjun Song, Sundong Kim, Minseok Kim, Jae-Gil Lee

Abstract: Neural networks can converge faster with help from a smarter batch selection strategy. In this regard, we propose Ada-Boundary, a novel adaptive-batch selection algorithm that constructs an effective mini-batch according to the learning progress of the model.Our key idea is to present confusing samples what the true label is. Thus, the samples near the current decision boundary are considered as the most effective to expedite convergence. Taking advantage of our design, Ada-Boundary maintains its dominance in various degrees of training difficulty. We demonstrate the advantage of Ada-Boundary by extensive experiments using two convolutional neural networks for three benchmark data sets. The experiment results show that Ada-Boundary improves the training time by up to 31.7% compared with the state-of-the-art strategy and by up to 33.5% compared with the baseline strategy.

Keywords: acceleration, batch selection, convergence, decision boundary

TL;DR: We suggest a smart batch selection technique called Ada-Boundary.

Data: [Fashion-MNIST](https://paperswithcode.com/dataset/fashion-mnist)