RoTaR: Efficient Row-Based Table Representation Learning via Teacher-Student Training (Short Paper)

Zui Chen, Lei Cao, Samuel Madden

Keywords: Table Representation Learning, Query Agnostic, Scalability

TL;DR: A table representation method that produces query agnostic embeddings

Abstract: We propose RoTaR, a row-based table representation learning method, to address the efficiency and scalability issues faced by existing table representation learning methods. The key idea of RoTaR is to generate query-agnostic row representations that could be re-used via query-specific aggregation. In addition to the row-based architecture, we introduce several techniques: cell-aware position embedding, AutoEncoder objective in transformer models, teacher-student training paradigm, and selective backward to improve the performance of RoTaR model.