Training Decision Trees as Replacement for Convolution Layers

Wolfgang Fuhl

Abstract:

We present an alternative layer to convolution layers in con-
volutional neural networks (CNNs). Our approach reduces
the complexity of convolutions by replacing it with binary
decisions. Those binary decisions are used as indexes to con-
ditional distributions where each weight represents a leaf in a
decision tree. This means that only the indices to the weights
need to be determined once, thus reducing the complexity of
convolutions by the depth of the output tensor. Index compu-
tation is performed by simple binary decisions that require
fewer cycles compared to conventionally used multiplica-
tions. In addition, we show how convolutions can be replaced
by binary decisions. These binary decisions form indices in
the conditional distributions and we show how they are used
to replace 2D weight matrices as well as 3D weight tensors.
These new layers can be trained like convolution layers in
CNNs based on the backpropagation algorithm, for which we
provide a formalization.
Our results on multiple publicly available data sets show that
our approach performs similar to conventional neuronal net-
works. Beyond the formalized reduction of complexity and
the improved qualitative performance, we show the runtime
improvement empirically compared to convolution layers.