Intriguing Properties of Learned Representations

Amartya Sanyal, Varun Kanade, Philip H. Torr

Abstract:

A key feature of neural networks, particularly deep convolutional neural networks, is their ability to learn useful representations from data. The very last layer of a neural network is then simply a linear model trained on these learned representations. Despite their numerous applications in other tasks such as classification, retrieval, clustering etc., a.k.a. transfer learning, not much work has been published that investigates the structure of these representations or indeed whether structure can be imposed on them during the training process.

In this paper, we study the effective dimensionality of the learned representations by models that have proved highly successful for image classification. We focus on ResNet-18, ResNet-50 and VGG-19 and observe that when trained on CIFAR10 or CIFAR100, the learned representations exhibit a fairly low rank structure. We propose a modification to the training procedure, which further encourages low rank structure on learned activations. Empirically, we show that this has implications for robustness to adversarial examples and compression.

Keywords: deep learning, low rank representations, adversarial robustness

TL;DR: Imposing a low rank structure on learned representations in deep networks yields a lot of interesting benefits.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/intriguing-properties-of-learned/code)