Image Classification Understanding with Model Inspector Tool

Flávio Arthur O. Santos, Maynara Donato de Souza, Pedro Oliveira, Leonardo Nogueira Matos, Paulo Novais, Cleber Zanchettin

Abstract: This paper proposes a novel method called U Analysis for interpreting the behavior of image classification models. The method allows the evaluation of the interdependence between patches of information in an image and their impact on the model’s classification performance. In addition, the paper introduces the Model Inspector tool that allows users to manipulate various visual features of an input image to understand better the model’s robustness to different types of information. This work aims to provide a more comprehensive framework for model interpretation and help researchers and practitioners better understand the strengths and weaknesses of deep learning models in image classification. We perform experiments with CIFAR-10 and STL-10 datasets using the ResNet architecture. The findings show that ResNet model trained with CIFAR-10 and STL-10 presents counter-intuitive feature interdependence, which is seen as a weakness. This work can contribute to developing even more advanced tools for analyzing and understanding deep learning models.