When and how convolutional neural networks generalize to out-of-distribution category-viewpoint combinations

Spandan Madan, Timothy Henry, Jamell Dozier, Helen Ho, Nishchal Bhandari, Tomotake Sasaki, Frédo Durand, Hanspeter Pfister, Xavier Boix

Abstract: The combination of object recognition and viewpoint estimation is essential for visual understanding. However, convolutional neural networks often fail to generalize to object category–viewpoint combinations that were not seen during training. The authors investigate the impact of data diversity and architectural choices on the capability of generalizing to out-of-distribution combinations.