Budget RNNs: Multi-Capacity Neural Networks to Improve In-Sensor Inference Under Energy Budgets

Tejas Kannan, Henry Hoffmann

Abstract:

Recurrent neural networks (RNNs) are well-suited
to the sequential inference tasks often found in embedded sensing
systems. While RNNs have displayed high accuracy on many
tasks, they are poorly equipped for inference under energy
budgets that are unknown at design time. Existing RNNs meet
energy constraints in sensor environments by training models
to subsample input sequences. The tight coupling between the
sampling strategy and the RNN prevents these systems from
generalizing to new energy budgets at runtime. To address this
problem, we present a novel RNN architecture called the Budget
RNN. Budget RNNs use a leveled architecture to decouple the
sampling strategy from the RNN model, allowing a single Budget
RNN to change its subsampling behavior at runtime. We further
propose a runtime feedback controller to optimize the model’s
accuracy for a given energy budget. Across a set of budgets,
the Budget RNN inference system achieves a mean accuracy
of roughly 3 points higher than standard RNNs. Alternatively,
Budget RNNs can achieve comparable accuracy to existing RNNs
while under 20\% smaller budgets.