Permutation Invariant Strategy Using Transformer Encoders for Table Understanding

Anonymous

Paper Link: https://openreview.net/forum?id=aWWJU0LhK1S

Paper Type: Long paper (up to eight pages of content + unlimited references and appendices)

Abstract: Representing text in tables is essential for many business intelligence tasks such as semantic retrieval, data exploration and visualization, and question answering. Existing methods that leverage pretrained Transformer encoders range from a simple construction of pseudo-sentences by concatenating text across rows or columns to complex parameter-intensive models that encode table structure and require additional pretraining. In this work, we introduce a novel encoding strategy for Transformer encoders that preserves the critical property of permutation invariance across rows or columns. Unlike existing state-of-the-art methods for Table Understanding, our proposed approach does not require any additional pretraining and still substantially outperforms existing methods in almost all instances. We demonstrate the effectiveness of our proposed approach on three table interpretation tasks: column type annotation, relation extraction, and entity linking through extensive experiments on existing tabular datasets.

Presentation Mode: This paper will be presented in person in Seattle

Copyright Consent Signature (type Name Or NA If Not Transferrable): Sarthak Dash

Copyright Consent Name And Address: IBM Research AI, Thomas J Watson Research Center, Yorktown Heights, NY