Does Geometric Structure in Convolutional Filter Space Provide Filter Redundancy Information?

Anshul Thakur, Vinayak Abrol, Pulkit Sharma

Keywords: Positional filter relevance, Geometric modelling, CNN filter redundancy

TL;DR: This paper analyses the convolutional layer filter space using simplical geometry to establish a relation between filter relevance and their location on the simplex.

Abstract: This paper aims to study the geometrical structure present in a CNN filter space for investigating redundancy or importance of an individual filter. In particular, this paper analyses the convolutional layer filter space using simplical geometry to establish a relation between filter relevance and their location on the simplex. Convex combination of extremal points of a simplex can span the entire volume of the simplex. As a result, these points are inherently the most relevant components. Based on this principle, we hypothesize a notion that filters lying near these extremal points of a simplex modelling the filter space are least redundant filters and vice-versa. We validate this positional relevance hypothesis by successfully employing it for data-independent filter ranking and artificial filter fabrication in trained convolutional neural networks. The empirical analysis on different CNN architectures such as ResNet-50 and VGG-16 provide strong evidence in favour of the postulated positional relevance hypothesis.