Color-Dependent Prediction Stability of Popular CNN Image Classification Architectures

Laurent Mertens, Elahe Yargholi, Jan Van den Stock, Hans P. Op de Beeck, Joost Vennekens

Abstract: The ImageNet-1k dataset has been a major contributor to the development of novel CNN-based image classification architectures over the past 10 years. This has led to the advent of a number of models, pre-trained on this dataset, that form a popular basis for creating custom image classifiers by means of transfer learning. A corollary of this process is that whatever weaknesses and biases the original model possesses, the derived model will also have. Some of these have already been extensively covered, but color sensitivity has so far been understudied. This paper explores the prediction stability of several popular CNN architectures when input images are subjected to hue or saturation shifts. We show that even small shifts in image hue can alter a model’s initial prediction, with larger shifts introducing changes up to 60% and 40% of the time for AlexNet and VGG16 respectively. For all models considered, saturation changes have less impact. To illustrate the issue being inherited by models obtained through transfer learning, we confirm that EmoNet, a model derived from AlexNet, exhibits similar behavior. By further comparing a same architecture trained separately on ImageNet-1k, Places365 and Stylized ImageNet, we confirm that the issue is shared across datasets. Finally, we propose a new preprocessing data augmentation to alleviate this problem.