Interpreting Image Super-Resolution in Artificial Neural Networks from Global and Local Views

Xiaochen Liu, Alexander W. Jacob, Wei Song, Antonio Liotta

Abstract: Work on image super-resolution (SR), to construct higher-resolution images starting from low-quality ones, has focused primarily on reconstruction algorithms and specific application domains. In this work, we aim at methods to aid interpreting SR inner-working, with a view to improve explain-ability. We propose a novel gradient-based attribution approach, to provide interpretations from global and local perspectives, dubbed glocal attribution map (GL-AM). After verification with five different SR models, we show that GL-AM: (1) is a powerful tool to understand the principles of SR networks from both global and local views; (2) provides the consensus and variation sensitivity of different models to the input; (3) is more effective to emphasize the features captured by the attention mechanism (for the SR model) through feature re-calibration; (4) is more computationally efficient and more effective as the region of interest increases.