RoTaR: Efficient Row-Based Table Representation Learning via Teacher-Student Training (Short Paper)Download PDF

Published: 21 Oct 2022, Last Modified: 16 May 2023TRL @ NeurIPS 2022 PosterReaders: Everyone
Keywords: Table Representation Learning, Query Agnostic, Scalability
TL;DR: A table representation method that produces query agnostic embeddings
Abstract: We propose RoTaR, a row-based table representation learning method, to address the efficiency and scalability issues faced by existing table representation learning methods. The key idea of RoTaR is to generate query-agnostic row representations that could be re-used via query-specific aggregation. In addition to the row-based architecture, we introduce several techniques: cell-aware position embedding, AutoEncoder objective in transformer models, teacher-student training paradigm, and selective backward to improve the performance of RoTaR model.
0 Replies