Differential Geometry Boosts Convolutional Neural Networks for Object Detection

Chu Wang

Abstract: Convolutional neural networks (CNNs) have had dramatic success in appearance based object recognition tasks such as the ImageNet visual recognition challenge. However, their application to object recognition and detection thus far has focused largely on appearance images as inputs. Motivated by demonstrations that depth can enhance the performance of CNN-based approaches, we consider the benefits of adding differential geometric shape features in a principled manner. This elementary idea of using zeroth order (depth), first-order (surface normal) and second-order (surface curvature) features boosts the performance of a CNN that has been pretrained on a color image database. In an object detection task involving 19 categories we improve on the current state-of-the-art detection accuracy on the NYUv2 dataset of 35.6% by Gupta et al. by 10.4% to a new result of 39.3%. Our results provide strong evidence that the abstraction of surface shape benefits object detection and recognition.