Deepsub: A Novel Subset Selection Framework for Training Deep Learning Architectures

Subhankar Banerjee, Shayok Chakraborty

Abstract: Deep learning algorithms automatically learn a set of informative features from a given dataset and have depicted commendable performance on a variety of computer vision applications. However, efficient training of deep learning architectures with a large number of hidden layers is mostly dependent on high-end GPUs and distributed computing infrastructures. Some applications (such as applications running on mobile platforms), have limited access to computational resources and face a fundamental challenge in handling large scale training data. Cloud services can be leveraged for training, but involve challenges with data privacy and cost. In such applications, it is crucial to select a subset of informative training samples and use only the extracted subset to induce the deep model. In this paper, we propose a novel subset selection algorithm, DeepSub, to address this practical challenge. Our framework is computationally efficient, easy to implement and also enjoys nice theoretical properties. Our extensive empirical studies on three challenging computer vision applications (face, handwritten digits and object recognition), using three popular deep learning architectures (AlexNet, GoogleNet and ResNet) corroborate the potential of DeepSub over competing baselines.