Deep processing of structured data

Łukasz Maziarka, Marek Śmieja, Aleksandra Nowak, Jacek Tabor, Łukasz Struski, Przemysław Spurek

Abstract:

We construct a general unified framework for learning representation of structured
data, i.e. data which cannot be represented as the fixed-length vectors (e.g. sets,
graphs, texts or images of varying sizes). The key factor is played by an intermediate
network called SAN (Set Aggregating Network), which maps a structured
object to a fixed length vector in a high dimensional latent space. Our main theoretical
result shows that for sufficiently large dimension of the latent space, SAN is
capable of learning a unique representation for every input example. Experiments
demonstrate that replacing pooling operation by SAN in convolutional networks
leads to better results in classifying images with different sizes. Moreover, its direct
application to text and graph data allows to obtain results close to SOTA, by
simpler networks with smaller number of parameters than competitive models.

Keywords: structured data, representation learning, deep neural networks

TL;DR: General framework of learning representation of structured inputs.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1810.01868/code)