Processing of incomplete images by (graph) convolutional neural networks

Tomasz Danel, Marek Śmieja, Łukasz Struski, Przemysław Spurek, Lukasz Maziarka

TL;DR: (Graph) convolutional neural networks applied to incomplete images without imputations

Keywords: missing data, graph convolutional neural networks, convolutional neural networks, image inpainting, imputation of incomplete images

Abstract: We investigate the problem of processing incomplete images by neural networks without replacing missing values. To deal with this problem, we first represent an image as a graph, in which missing pixels are entirely ignored. The graph image representation is processed using Geo-GCN -- a type of graph convolutional neural networks, which is a proper generalization of classical CNNs operating on images. On one hand, our approach avoids the problem of missing data imputation while, on the other hand, there is a natural correspondence between CNNs and Geo-GCN. Experiments confirm that our approach performs better than analogical CNNs with the imputation of missing values on typical classification and reconstruction tasks.