Selectivity metrics can overestimate the selectivity of units: a case study on AlexNet

Ella M. Gale, Anh Nguyen, Ryan Blything, Nicholas Martin and Jeffrey S. Bowers,

Abstract: Various methods of measuring unit selectivity have been developed in order to understand the representations learned by neural networks (NNs). Here we undertake a comparison of four such measures on AlexNet, namely, localist selectivity, \precision (Zhou et al, ICLR 2015), class-conditional mean activity selectivity CCMAS; (Morcos et al, ICLR 2018), and a new measure called top-class selectivity. In contrast with previous work on recurrent neural networks (RNNs), we fail to find any 100\% selective `localist units' in AlexNet, and demonstrate that the \precision and CCMAS measures provide a much higher level of selectivity than is warranted, with the most selective hidden units only responding strongly to a small minority of images from within a category. We also generated activation maximization (AM) images that maximally activated individual units and found that under (5\%) of units in fc6 and conv5 produced interpretable images of objects, whereas fc8 produced over 50\% interpretable images. Furthermore, the interpretable images in the hidden layers were not associated with highly selective units. These findings highlight the problem with current selectivity measures and show that new measures are required in order to provide a better assessment of learned representations in NNs. We also consider why localist representations are learned in RNNs and not AlexNet.

Keywords: AlexNet, neural networks, selectivity, localist, distributed, represenataion, precision, measures of selectivity, object detectors, single directions, network analysis

TL;DR: Common selectivity metrics overestimate the selectivity of units, true object detectors are extremely rare, but class selectivity does increase with depth.