Embedding File Structure for Tabular File Preparation

Gerardo Vitagliano, Mazhar Hameed, Alejandro Sierra Múnera, Felix Naumann

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: representation, tabular embedding, file structure, data preparation, table representation learning

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose a novel architecture to encode the structure of tabular files and apply it to address data preparation tasks.

Abstract:

We introduce the notion of file structure, the set of characters within a file's content that do not belong to data values.
Data preparation can be considered as a pipeline of heterogeneous steps with the common theme of wrangling the structure of a file to access its payload in a downstream task.
We claim that solving typical data preparation tasks benefits from an explicit representation of file structure.
We propose a novel approach for learning such a representation, which we call a structural embedding, using the raw file content as input.
Our approach is based on a novel neural network architecture, composed of a transformer module and a convolutional module, trained in a self-supervised fashion on almost 1M public data files to learn structural embeddings.
We demonstrate the usefulness of structural embeddings in several steps of a data preparation pipeline: data loading, row classification, and column type annotation.
For these tasks, we show that our approach obtains performances comparable with state-of-the-art baselines on six real-world datasets, and, more importantly, we improve upon such baselines by combining them with the structural embeddings provided by our approach.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7741