Doubly Nested Network for Resource-Efficient Inference

Jaehong Kim, Sungeun Hong, Yongseok Choi, Jiwon Kim

Abstract: We propose a new anytime neural network which allows partial evaluation by subnetworks with different widths as well as depths. Compared to conventional anytime networks only with the depth controllability, the increased architectural diversity leads to higher resource utilization and consequent performance improvement under various and dynamic resource budgets. We highlight architectural features to make our scheme feasible as well as efficient, and show its effectiveness in image classification tasks.

TL;DR: We propose a new anytime neural network which allows partial evaluation by subnetworks with different widths as well as depths.

Keywords: Doubly Nested Network, Anytime neural network, Resource-Efficient Inference, Dynamic resource budgets