Differential Geometry Boosts Convolutional Neural Networks for Object Detection

Chu Wang, Kaleem Siddiqi

Abstract: Convolutional neural networks (CNNs) have had dramatic success in appearance based object recognition tasks such as the ImageNet visual recognition challenge [8]. However, their application to object recognition and detection thus far has focused largely on intensity or color images as inputs. Motivated by demonstrations that depth can enhance the performance of CNN-based approaches [2][5], in this paper we consider the benefits of also including differential geometric shape features. This elementary idea of using zeroth order (depth), first-order (surface normal) and second-order (surface curvature) features in a principled manner boosts the performance of a CNN that has been pretrained on a color image database. Notably, in an object detection task involving 19 categories we achieve 39.30% accuracy on the NYUv2 dataset, which is a 10.4% improvement over the current state-of-the-art accuracy of 35.6% using the method in [5]. In the simpler scenario of turntable style object recognition, our experiments on the University of Washington (UW) RGB-D dataset yield an accuracy of 88.7% correct recognition over 51 object categories, where the best competing result is 87.5% [2]. Taken together, our results provide strong evidence that the abstraction of surface shape benefits object detection and recognition.