ShapeEmbed: a self-supervised learning framework for shape quantification

Anna Foix Romero, Craig Russell, Alexander Krull, Virginie Uhlmann

TL;DR: ShapeEmbed is a self-supervised method that uses Euclidean distance matrices to learn descriptors of the shape of 2D objects in images that is invariant to irrelevant geometric transformations.

Abstract: The shape of objects is an important source of visual information in a wide range of applications. One of the core challenges of shape quantification is to ensure that the extracted measurements remain invariant to transformations that preserve an object’s intrinsic geometry, such as changing its size, orientation, and position in the image. In this work, we introduce ShapeEmbed, a self-supervised representation learning framework designed to encode the outline of objects in 2D images into a shape descriptor that is invariant to translation, scaling, rotation, reflection, and outline point indexing. ShapeEmbed relies on a Euclidean distance matrix representation of the outline of input objects. Our approach overcomes the limitations of traditional shape descriptors while improving upon existing state-of-the-art autoencoder-based approaches. We demonstrate that the descriptors learned by our framework outperform their competitors in shape classification tasks on natural and microscopy images. Our framework is also generative, thus allowing for sampling and full reconstruction of 2D outlines from their latent feature vectors.

Primary Area: General Machine Learning->Representation Learning

Keywords: shape quantification, shape descriptors, VAE, generative model, representation learning, microscopy

Submission Number: 4954