Share or Split : which is more efficient?

Yinpeng Chen, Zicheng Liu, Lijuan Wang, Zhengyou Zhang

Abstract: In this paper, we investigate two different feature representations in convolutional neural networks (CNN): (a) Share - all classes share the same feature space, and a fully connected layer is used to decode class activation, and (b) Split - each class has its own feature space and a class is activated if its corresponding feature vector has a large norm. This is inspired by Capsules (\cite{Sabour17capsules}) which splits the feature space. We compare these two representations on a transformed MNIST dataset, which adds random scales and translations on the original digits. The experimental results show that Split has better performance when data is limited, while Share is better when data is big.