DHOG: Deep Hierarchical Object Grouping

Luke Nicholas Darlow, Amos Storkey

Keywords: Unsupervised learning, Deep neural networks, clustering

Abstract: Unsupervised learning of categorical representations using data augmentations appears to be a promising approach and has proven useful for finding suitable representations for downstream tasks. However current state-of-the-art methods require preprocessing (e.g. Sobel edge detection) to work. We introduce a mutual information minimization strategy for unsupervised learning from augmentations, that prevents learning from locking on to easy to find, yet unimportant, representations at the expense of more informative ones requiring more complex processing. We demonstrate specifically that this process learns representations which capture higher mutual information between augmentations, and demonstrate that these representations are better suited to the downstream exemplar task of clustering. We obtain substantial accuracy improvements on CIFAR-10, CIFAR-100-20, and SVHN.

One-sentence Summary: A deep clustering technique that uses mutual information maximisation for multiple diverse solutions, thereby improving the performance on the original mutual information-based unsupervised objective.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=x51yDzGlqX