How to Transform Kernels for Scale-Convolutions

Ivan Sosnovik, Artem Moskalev, Arnold W.M. Smeulders

Keywords: Scale, Equivariance, Dilation

TL;DR: We demonstrate that the equivariance error affects the performance of equivariant networks. We introduce an approach to rescale a basis in scale-convolution, so the equivariance error is minimized.

Abstract: Scale is often seen as a given, disturbing factor in many vision tasks. When doing so it is one of the factors why we need more data during learning. In recent work scale equivariance was added to convolutional neural networks. It was shown to be effective for a range of tasks. We aim for accurate scale-equivariant convolutional neural networks (SE-CNNs) applicable for problems where high granularity of scale and small kernel sizes are required. Current SE-CNNs rely on weight sharing and kernel rescaling, the latter of which is accurate for integer scales only. To reach accurate scale equivariance, we derive general constraints under which scale-convolution remains equivariant to discrete rescaling. We find the exact solution for all cases where it exists, and compute the approximation for the rest. The discrete scale-convolution pays off, as demonstrated in a new state-of-the-art classification on MNIST-scale and on STL-10 in the supervised learning setting.